Embed Size (px)
Citation preview
WASC ARCPRE-CONFERENCE WORKSHOP
ONLINE COURSE QUALITY: IDENTIFYING THE WHAT’S AND HOW’S (AND THE WHY’S)
ERRIN HEYMAN, ED.D.DEAN, SCHOOL OF EDUCATIONJONES INTERNATIONAL UNIVERSITY
Description
Online courses and programs are not new news, and their numbers continue to grow in higher education. There remain questions and
concerns, however, around quality and efficacy, perhaps even more now than ever as Competency Based programs are becoming more
prevalent.
This workshop allows participants to explore general principles and specific strategies for approaching instructional design (with quality and the credit hour in mind); assessment in online courses; and creating a university experience at a distance. The workshop will provide examples of quality issues such as evaluation of threaded discussions, ‘build or buy’ scenarios, and other relevant issues raised by workshop participants. Participants should come prepared to identify an issue they are working on; in small groups, guided by the facilitator, participants will work though potential solutions.
Agenda
Introductions [15 mins] Measuring Quality (presentation: Credit Hours, Online ‘Seat
Time’) [40 mins w/ 5 mins ‘text’ break] Activity [20-25 mins] Snack Break [9:30, 20 mins]
Defining Online Quality (presentation) [15 mins] w/ Activity [15 mins]
‘Text’ Break [5 mins] Pre-Presentation Activity [15 mins]
Creating the Quality University-Wide Experience—virtually (presentation) [15 mins]
Q & A (plus ‘parking lot’) [15 mins]
Introductions
Who are you? (name, institution, role)
What is your institution’s online presence?
(Full online [courses/programs], blended, web-enhanced)
Why are you here?
Measuring Credit
What is the Carnegie Unit? The Carnegie Unit and Credit Hour are time-based
references for measuring educational attainment used by American universities and colleges.
A Carnegie “Unit” assesses secondary school attainment.
A Student Hour derived from the Carnegie Unit (commonly called a credit hour) assesses post-secondary school attainment.
A Credit Hour defined as the equivalent of one clock hour (50 minutes) of lecture time for a single student per week over the course of a semester, which equates to approximately 1/10th of a Carnegie Unit.
Measuring Credit
What is the current definition of a Credit Hour based on the Carnegie Unit?
One clock hour of instruction per week
Expectation of two hours of out of class work each week
Over the course of a semester/trimester/quarter
Measuring Credit
A Carnegie Unit “Fun Fact”:
Did you know that back in 1906, the Carnegie Unit was originally designed as a means to evaluate the credit that a faculty member should achieve to establish their pension funds?
WHAT IS A CREDIT HOUR?
Clock Hours? Lecture Hours? Lab Hours? Externship Hours? Clinical Hours? Student Hour (Carnegie
Unit)?
1
Many Different Definitions
Many Different Measurements
Accrediting AgencyState AgencyLicensing BoardsFederal Financial Aid
Foreign Systems (ECTS)
Defining Online Credit Hour?CONCERNS Current definitions based on
instructional time do not apply to online
Many programs/courses are blended Seat time and seat time equivalences
may not be the appropriate measure What about the two hours “outside” of
class?
Federal Definition
34 CFR 600.2Credit Hour: Except as provided in 34 CFR
668.8(k) and (l), a credit hour is an amount of work represented in intended learning outcomes and verified by evidence of student achievement that is an institutionally established equivalency that reasonably approximates not less than –
Federal Definition
(1) One hour of classroom or direct faculty instruction and a minimum of two hours of out of class student work each week for approximately fifteen weeks for one semester or trimester hour of credit, or ten to twelve weeks for one quarter hour of credit, or the equivalent amount of work over a different amount of time; or
(2) At least the equivalent amount of work as required in paragraph of this definition for other academic activities as established by the institution including laboratory work, internships, practica, studio work, and other academic work leading to the award of credit hours.
But….
Deference to Accrediting Agencies34 CFR 602.24
Accrediting Agency must do an “effective review and evaluation of the reliability and accuracy of the institution’s assignment of credit hours.”
Accrediting agency must make “a reasonable determination …[that] the institution’s assignment of credit hours conforms to commonly accepted practice in higher education.”
CRITICISMS OF THE RELIANCE ON SEAT TIME
Seat time seems like an arbitrary unit and does not measure the “amount of work represented in intended learning outcomes and verified by evidence of student achievement” (34 CFR 600.2) Level of rigor and workload varies in a 3 credit lower
division course from that required in a 3 credit upper division course
Level of Achievement varies from program to program (Requirement of a C or above in some classes)
Level of attendance varies based on student absences
…Criticisms
Seat time is not a measure of student achievement (Some students still fail even if they are physically present)
Student achievement of intended learning outcomes can be a professional judgment, not always an objective measurement
ACE, CLEP, DANTES, Portfolio Assessment Credits, Competency Based Ed, Direct Assessment are not directly based on hours.
CRITICISMS OF THE RELIANCE ON EQUIVALENCIES TO SEAT TIME IN ONLINE COURSES
“Amount of work” varies by student Some students require less work to achieve
intended learning outcomes Some instructors require less work
Student achievement varies by instructor and student
Amount of work and student achievement varies by school
MEANS TO DETERMINE AN “EQUIVALENT AMOUNT OF WORK” AND ASSESS STUDENT ACHIEVEMENT OF LEARNING OUTCOMES
Review demonstration of skills learned and used by each student in class participation activities
Review student projects that demonstrate skills learned and applied by each student
Assess student’s ability to demonstrate research skills in the subject matter
Assess student achievement of learning outcomes
Evaluate personal growth in student attitudes toward the appreciation of diversity in the learning community
Online Development Considerations(instructional design)
The average time spent on a “screen” (multiple screens comprise of an entire course) – this is generally accepted at being between 3-5 minutes per screen.
Media assets (self playing), videos or animation – the calculation for this is generally a 1:1 ratio although for calculation for learning needs to be examined carefully as many media assets (audio and video) may need to be listened to multiple times
(diagram, courtesy: Karuna Sanghvi
– Coolwords, Weblog)
Course development/Instructional design
Links to external learning assets – calculated as an average of the time required to ‘consume’ content (i.e. average time to read an article or watch a video).
Simulations and interactions, calculated as a factor of the complexity of the interaction or simulation and can be very tricky as absorption of material can vary significantly depending on the learner profile and capability – the best calculation is to develop an average based on the specific interaction and expertise of the instructor on the requirement.
Assignments – dependent on the assignment and the time allocated for completion based on instructor and course requirement.
Student-teacher interactions – online information exchange if it is tracked and managed as a part of the learning activity.
Assessments and self-tests, all a part of overall seat time developed as an average for student use.
(criteria developed in conjunction with emantras, inc.)
Determining Seat Time
Example:West Coast University Catalog description: Calculation of credit hours in an online or blended format course is based
on the consideration of the following activities: The time spent in live instruction. Live instruction may be
synchronous or instructor-led online time, or residential learning in the classroom (blended coursework)
The number of screens viewed in the course of online instruction. The average time spent on a “screen” is generally calculated as being between 3-5 minutes per screen.
The run-time for required media asset assignments calculated on a 1:1 ratio to seat time. Run time may be factored to account for expected multiple viewings of the asset for review and re-enforcement of the material. Assets may include self playing videos or animation, or audio podcasts and recordings
Determining Seat Time
West Coast University Catalog description (cont):
Links to external learning assets – calculated as an average of the time required to consume content such as by reading an article watching a self-paced instructional video, playing an instructional game or completing a simulation
Assignments – The instructor expectation of time spent in online instructional (not homework) assignments and activities such as: postings to group discussion sites/bulletin boards. online group project work use of class social media sites for group
discussion/participation student-teacher interaction
Content in Action
Examples from a WCU Course
22
Week 1 – main pageWeek 1 contains the following items for student review. To view the presentation, click on Week 1 Presentation.
23
Week 1 PresentationHere is topic 1.1 of the presentation. If students view the “ON” icon animated, then the voice-over narrative of the presentation is active.
The icons in all presentations are:
•Aa = Glossary•Notepad = notes•Microphone = transcript•Volume toggle = sound on or off•Pause, play, previous, next and replay
Navigational breadcrumbs. (“Home” is student’s Angel Learning account.)
24
Week 1 Presentation
25
Week 1 Case Study
The Case Study is an interactive self-check tool of materials reviewed so far. Self-check items are not graded.
Click on Enter to view the Case Study.
Students will be given a scenario and be tested on it.
Click Test Me.
26
Week 1 Case StudyMost Case Studies have 3-4 questions.
Choose the answer and submit.
Click Next until the end of the Case Study.
Correct answer!
Incorrect answer! (correct answer
given)
27
Week 1 FlashcardsFlashcards are also a self-check tool. They appear in most courses and are also interactive.
Review the directions and click Start.
28
Week 1 Discussion
The screen is divided into two sections:
the top section contains the directions and the bottom section contains the forum.
Forum with
posts.
Directions for discussion.
Week1 Discussion 1 topic and question.
29
Week 1 Drop-box assignmentAs with discussion forums, drop-box assignments are also graded items.
Click on the drop-box item to view the assignment.
30
Week 1 Dropbox assignmentReview the directions for the assignment.
Submit the assignment by uploading it.
Directions for this assignment.
Submit assignment here.Grades and instructor comments are also
seen here.
31
Week 3, Quiz 1Quizzes are graded items also. The number of quizzes in each week depends on the course.
Here, the first and second quiz for this course appears in week3.
The quiz is currently disabled because the term has ended for this course. Otherwise students would click on “Begin Now” and take the quiz.
On this page, information about the quiz (time length, number of attempts, maximum points possible) are also stated.
If a student submits the quiz, their grade and date of submission will also be posted here.
Questions Remaining…
Is the Credit Hour Obsolete?
(if so) how can we better measure student learning and achievement of learning outcomes?
Questions Remaining…
As online is integrated into more student programs, how will the U.S. Department of Education effectively assess credit hour equivalencies?
Think about recent activity:
MOOCs for credit (ACE) Several institutions now approved under Direct Assessment approach from USDE.
Templates
Credit hour— “Carnegie Unit Report”
In and out of class template
Templates--example
Academic Credit Analysis Calculations
Out of Class Hours * ALL calculations are per Course
15 - Lecture Contact Hours Total SEMESTER Credits 45 Contact Hours Total Number of Hours
Standard Contact Hours 1 Semester Credit 30 - Lab Contact Hours
45 - Cl inica l Contact Hours
Total Number of HoursTotal CONTACT
Hours Out of Class Hours
15 - Lecture Contact Hours
*BRN Standard Contact Hours 1 Semester Credit 45 - Lab Contact Hours *
45 - Cl inica l Contact Hours
Semester Credit ConversionsLecture Contact Hours Semester Lecture Credi t Lecture Credits
15
Lab Contact Hours Semester Lab Credi t Lab Credits
30
Cl inica l Contact Hours Semester Cl inica l Credi t Clinical Credits
45
TOTAL Semester Credits
*45 - Lab Contact Hours ONLY appl ies to Nurs ing Courses .
[ Lecture credit + Lab credit + Clinical credit = TOTAL Semester Credits (per course)] :
Templates—example, on-ground
Course Number English 140Course Title Written Communication
Credits Awarded 3Lecture Hours 45
Lab Hours 0Clinical Hours 0
Total In Class Hours 45
Out of Class Assignments Hours Expected to be Devoted to this Assignment
Week 1: Read chapters 1-2 and 21-24 (5 hrs.), Complete Review Exercises in Each Chapter (1 hr.), Writing Assignment and Revision (3 hrs.) Research Project – Topic Selection (3 hrs.)
12
Week 2: Read chapters 3-5 (3 hrs.), Read Essay and Complete Narrative Writing Assignment (3 hrs.), Topic sentence exercises (1 hr.) Internet Search Form (2 hrs.)
9
Week 3: Read chapters 6-8 (3 hrs.), APA Manual chapter 6 (1 hr.), Read Exemplification Essay and Complete Writing Assignment (3 hrs.), Research Project – Locate Sources and Complete Annotated Bibliography (4 hrs.),
11
Week 4: Read chapters 15-17 (3 hrs.), Complete and Revise Process Essay (3 hrs.), Research Project – Thesis Statement and Outline (4 hrs.) Study for Mid Term Exam (3 hrs.)
13
Week 5: Read chapters 18-20 (3 hrs.), Film Review and Writing Assignment (4 hrs.), Research Project – Locate Additional Sources (3 hrs.)
10
Week 6: Read chapters 35-38 (4 hrs.) Revise Thesis Statements (1 hr.), Research Project – Complete First Draft (5 hrs.)
10
Documenting the Carnegie Unit
Templates—example, onlineb. Assignments – The instructor expectation of time spent in online instructional assignments:
Homework assignments:Week 1 – 1 hoursWeek 2—2 hours assignment; quiz (and prep)- 1.5Week 3—2 writing assignments, 4 hoursWeek 4—2 hours; quiz (and prep) -1.5 hours.Week 5—2 hours; quiz (and prep) -1.5 hours.Week 6—2 writing assignments, 4 hoursWeek 7—2 writing assignments, 4 hoursWeek 8—2 writing assignments, 4.5 hoursWeek 9—2 hours; quiz (and prep)- 1.5 hours.
1. postings to group discussion sites/bulletin boards.N/A
2. online group project workN/A
3. use of class social media sites for group discussion/participationN/A
4. student-teacher interaction/tutoring timeEstimated average of 1 hours per student
Total Hours of Out of Class Work 90Total Hours (In-Class and Out-of-Class) 135
Online Quality
What are the concerns? Assessment in courses Assessment of courses
Concerns--Integrity
How do I know the student is who s/he says?How do I keep students from cheating (or ‘collaborating’ on work? Or using their books, or…?)How do I address the general outlook that online education is inferior to f2f?
“Forcing” Quality
Assessment In… Policies and procedures: threaded discussionsRubric(s)Link to outcomes
Attendance requirements (not just logging in)
Balance exams with written work/other requirements
“Forcing” Quality
Assessment Of… Evaluate instructors
Policies/procedures around interaction with threads, response times, feedback on assignments
Instructor vs. Vendor/Publisher creation
Example: Discussion RubricGrading Threaded Discussions
Student discussion grades will be based upon the below Rubric:
CATEGORY Points: _25_ Points: 15 Points: _5__ Points:_0 Topic Response
Responding to the instructor's topic by Wednesday.
Responding to the instructor's topic by Sunday.
Not responding to instructor’s topic.
Not responding to the instructor's topic.
Peer Response
Responding to 2 or more peer postings by Sunday.
Responding to 2 peer postings by Sunday.
Responding to 1 peer posting by Sunday.
Responding to NO peer postings.
Quality Threads
Student comments that add significantly to the discussion by suggesting other solutions, pointing out problems, or even respectfully disagreeing. Student also substantiates any comments made with reasoning or even source citation. A quality posting is ~75 words.
Student comments that add moderately to the discussion by suggesting other solutions, pointing out problems, or even respectfully disagreeing. Student does not substantiate any comments made with reasoning or even source citation. Posting is between 50-75 words.
Student comments that do not add to the discussion. Student does not substantiate any comments made with reasoning or even source citation. Posting is simple: "I agree" or "Yes" or "No"
Student does not participate at all in the threaded discussion.
Example: Participation vs. Attendance
Participation Policy: Participation is different from
attendance. Attendance shows “presence.” Participation, however, demonstrates understanding of content and relevance and connections among the content, peer discussions, and other class activities. Participation is gauged primarily through weekly threaded discussions.
Example: Netiquette
Disruptive Classroom Behavior: The online forum is a type of classroom; therefore, behavior
that persistently or grossly interferes with class activities is considered disruptive behavior and may be subject to disciplinary action. Such behavior inhibits other students' ability to learn and instructors' ability to teach. A student engaged in disruptive or inappropriate behavior may be asked to discontinue posting in threaded discussions, pending discussion and resolution of the concerns and may be reported to the Office of the Dean/Director or Academic Dean.
The same rules apply online as they do in person. Please be respectful of your instructor and other students. Foul discourse will not be tolerated. Be sure to read understand the following link concerning "netiquette."
http://www.albion.com/netiquette/corerules.html
Creating the University Experience
Importance of academic and social integration in persistence models (Bean, Tinto). (Woodley, 2004)
Course experience and presence—we now know. (Creating a Community of Learners)
Now it’s the ancillary experiences—student groups, student union, advisors and counselors, etc.
What ever happened to Second Life?
Woodley, A. (2004). Conceptualizing student dropout in part-time distance education: Pathologizing the normal? Open Learning, 9(1), 47-63.
Student Integration Model
Vincent Tinto proposed the Student Integration Model around five student characteristics: attitudes and biases held prior to entering higher education, commitment to goals, prior experiences in school, social and academic integration, and outcome orientation.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). Chicago: University of Chicago Press.
Considerations
Access to Services and Departments Bookstore Library Applying & Registering Financial Aid Tutoring/Academic Advising Counseling Student Union (what would this look like?) Clubs/Support Groups Career Services Campus Tours (virtual tours) Helpdesk/online support (given) Others?
Q & A
“Parking Lot” Additional questions/comments
