06T06 Checklist for Evaluating-Online-Courses

7/30/2019 06T06 Checklist for Evaluating-Online-Courses

1/16

November 2006

SouthernRegionalEducationBoard

592 10th St. N.W.Atlanta, GA 30318(404) 875-9211

www.sreb.org

Checklist forEvaluating Online

CoursesEducational Technology

Cooperative


2/16

This publication was produced by SREBs Educational Technology Cooperative and funded by the BellSouth

Foundation. For more information, e-mail [email protected] or call (404) 875-9211.


3/16

This checklist is based on the SREB report Standards for Quality Online Coursesand

is designed to assist states and schools in determining the quality and effectiveness of

online courses. It is suggested that each course be rated on the extent to which it meets

the criteria, with 3 indicating that a course does not meet the criteria and 1 indicating

that it does. This evaluation is key to ensuring that online courses provide students with

access to quality instruction and resources.

Introduction

Checklist for EvaluatingOnline Courses

DIRECTIONS: Rate each item based on the extent to which the item meets the criteria.

Circle the value.

Course:

Course Provider:

URL:

Reviewer:

E-mail:

School/Office:

Phone:

Date:

ConsiderationsTo what extent does the coursemeet the criteria in this area?

1. The course goals andobjectives are measurableand clearly state what theparticipants will know orbe able to do at the endof the course.

Objectives are matched tocontent requirements and tothe grade and skill levels ofthe intended audience.

Section A: Content

Academic Content Standards and Assessments

Course Overview and Introduction

Legal and Acceptable Use Policies

Teacher Resources

Meetscriteria

Partiallymeets

criteria

Does notmeet

criteria

Notenough

information Comments

1 2 3


4/16

2


2. The course content andassignments are aligned

with the states content

standards or nationallyaccepted content stan-dards set for AdvancedPlacement courses, tech-nology, computer science,or other courses whosecontent is not includedin the state standards.

Course objectives, studentlearning outcomes andassignments are listed and

align with establishedcontent standards and thecontent and format require-ments of required localschool system, state ornational assessments.

Meetscriteria

Partiallymeets

criteria

Doesnot meetcriteria

Notenough


1 2 3

3. Course tasks and assess-ments align with therequired local, state andnational assessments thatare associated with thecourse.

Advanced Placement testingrequirements; nationalassessments; state assess-ments; local school systemassessments; technologycertification assessments.

1 2 3

4. The course contentand assignments are ofsufficient rigor, depthand breadth to teachthe standards beingaddressed.

Clear and explicit alignmentexists between objectives,assessments, instructionalstrategies, content andtechnology; and the content,activities and assignmentsprovide multiple learningopportunities to master thestandards.

1 2 3

5. Information literacy andcommunication skills areincorporated and taughtas an integral part of the

curriculum.

Promotes student-to-studentcommunication as part oflesson activities; utilizescommunication forums

such as threaded discussionforums, chats, etc.; informa-tion literacy is addressed.

6. Sufficient learningresources and materials toincrease student successare available to studentsbefore the course begins.

Self-assessments or checkliststhat assess prerequisiteknowledge and skills; orien-tations, tutorials, activities,policies and procedures tohelp students acquire theprerequisite knowledge andskills.

1 2 3

1 2 3


5/16


7. A clear, complete courseoverview and syllabus areincluded in the course.

The syllabus and overviewinclude: course objectivesand student learning out-

comes; assignments; studentexpectations; time require-ments; required materials;the grading policy; teacher-student, teacher-parent con-tact policies; the intendedaudience; and the contentscope and sequence. Otherimportant informationincludes a rationale indicat-ing how this course is relatedto other courses, the date oforiginal publication and lastrevision, the instructor avail-ability and response time,

and a description of the finalproduct or other activitiesthat are central to the coursecontent.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

8. Course requirements areconsistent with coursegoals, are representativeof the scope of the courseand are clearly stated.

The course requirementsinclude: a timeframe for par-ticipation; an approximatetime required for individualactivities; expectations forcommunications, such asonline discussion, synchro-nous chats and e-mail; activi-ty and other assignmentexpectations; and survey

and/or exam expectations.

1 2 3

9. Information is providedto students, parents andmentors on how to com-municate with the onlineteacher and course pro-vider, including informa-tion on the process forthese communications.

This could be included inthe course materials or in theorientation to the coursematerials.

1 2 3

10. Issues associated withthe use of copyrightedmaterials are addressed.

The course provider guaran-tees in writing that all coursematerials that they developedcomply with copyright laws.

11. Academic integrity andnetiquette (Internetetiquette) expectationsregarding lesson activi-ties, discussions, e-mailcommunications andplagiarism are clearlystated.

The course includes guide-lines for students on the useof copyrighted materials,appropriate use of networkedresources and academichonesty.

1 2 3

1 2 3


6/16

4


12. Privacy policies are clearlystated.

This could be included inthe course materials or inthe orientation to the course

materials.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

13. Instructor resources andnotes are included.

Instructor has included:additional guidance forteachers; possible scenariosfor course module sequence;suggestions for guiding stu-dents on paths; teachingstrategies for diverse stu-dents; suggestions for foster-ing student interaction andparticipation in discussions;additional content materials,such as optional simulatedlab materials or activities or

extra-credit assignments;ideas for sensitivity to back-ground and student needs;and guidelines for using thelearning paths.

1 2 3

14. Assessment and assign-ment answers and expla-nations are included.

Built-in course assessmentsare provided, and user accessto answers and informationabout each item is available.

1 2 3


7/16

Section B: Instructional Design

Instructional and Audience Analysis

Course, Unit and Lesson Design

Goals/Objectives

Instructional Strategies and Activities

Communication and Interaction

Resources and Materials


1. Course design reflects aclear understanding ofstudent needs and incor-porates varied ways to

learn and multiple levelsof mastery of the curri-culum.

A course for students withlow literacy skills may bedesigned differently than acourse for students who are

taking Advanced Placement;for example, more scaffold-ing and chunking of con-tent, or more audio scriptsof written materials.

Meetscriteria

Partiallymeets

criteria


Notenough


12 3

2. The course is organizedinto units and lessons.

The course is organized byunits and the associatedlessons.

1 2 3

3. The course unit overviewdescribes the objectives,activities and resourcesthat frame the unit.

The unit overview: describesthe objectives, lessons andresources included in theunit; and includes a descrip-tion of the activities and

assignments that are centralto the unit.

1 2 3

4. Each lesson includes alesson overview, contentand activities, assign-ments and assessmentsto provide multiplelearning opportunitiesfor students to masterthe content.

The lesson overview:describes the objectives,activities, assignments,assessments, estimatedtimeframe, and resourcesincluded in the lesson.

1 2 3

5. The course is designed toteach concepts and skillsthat students will retain

over time.

Course includes: backwarddesign (focus on expectationsfor student learning); essen-

tial questions; broad-basedconcepts; important andworthwhile concepts or skillsthat are relevant to subjectmatter and to the real

world in which the contentmay be applied; and mean-ingful and authentic learningexperiences that help stu-dents apply course conceptsand achieve course objectives.

1 2 3


8/16

6


6. The course instructionincludes activities thatengage students in active

learning.

For students who need orwant remediation or enrich-ment, additional learning

activities are present;resources are present whenstudents need more founda-tional skills or explanationsfor very difficult concepts.The course also affords theopportunity for students tolearn at their own pace, withclear directions and structureas to expectations and time-lines with teacher input andmonitoring.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

7. Instruction providesstudents with multiple

learning paths to masterthe content, based onstudent needs.

Students are given choicesfor alternative assignments

and assessments to make thebest use of their talents andskills, as appropriate.

1 2 3

8. The teacher engagesstudents in learningactivities that address avariety of learning stylesand preferences.

Learning activities mayinclude: examples; case stud-ies; simulations; research; avariety of reading and writ-ing activities; visuals andgraphics; discussions andlabs; podcasts; streamingvideo; and audio-scriptedtext. A learning style inven-tory is included. Strategiesthat consider the different

learning styles of studentsare used.

1 2 3

9. The course providesopportunities for studentsto engage in higher-orderthinking, critical reason-ing activities and thinkingin increasingly complex

ways.

Supporting opinions withevidence from test andexperience; higher-orderthinking inquiries with avariety of questions; inter-personal and self-directionalskills; critical and systemsthinking (understandingthe interconnections amongsystems); activities thatrequire students to frame,analyze and solve problems,

form opinions and developmental models.

1 2 3

10. The course reflects multi-cultural education and isaccurate, current and freeof bias.

Images; language and contentand selection of print andnon-print resources.

1 2 3


9/16


11. The teacher can adaptlearning activities toaccommodate students

needs.

Students with disabilities;504 plans; ELL; low skill orliteracy levels; enrichment

and gifted.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

12. Readability levels, writtenlanguage assignments andmathematical require-ments are appropriate forthe course content andthe students.

Sensitivity to readabilityissues, written language skillsand mathematical require-ments is demonstrated.

1 2 3

13. The course design pro-vides opportunities forappropriate instructor-student interaction,including timely andfrequent feedback about

student progress.

Teachers respond to studentsin a timely fashion. Feedbackmay include e-mails, discus-sions and portfolios; feed-back using LMS tools (e.g.,dropbox, quizzes and grades;

phone calls; webinars; regularprogress reports; and end-of-semester/term grades) alsomay be used.

1 2 3

14. The course providesopportunities for appro-priate instructor-studentand student-studentinteraction to fostermastery and applicationof the material and aplan for monitoring thatinteraction.

Collaborative learningopportunities are availablethrough e-mail, discussionstrands, oral assessments,synchronous chats, web-inars, simulations, labactivities and other groupprojects.

1 2 3

15. The course providesopportunities for appro-priate student interaction

with the content to fostermastery and applicationof the material.

Course structure should bedesigned to facilitate, to thegreatest extent possible, stu-dent interaction with othersand with learning resources.

1 2 3

16. Students have access toresources that enrich thecourse content.

Materials and/or Web linkshave been reviewed forappropriateness and arealigned with course objec-tives and specifications.

1 2 3


10/16

8

Section C: Student Assessment

Evaluation Strategies

Adequate and Appropriate Methods and Procedures

Feedback

Assessment Resources and Materials


1. Student evaluation strate-gies are consistent withcourse goals and objec-tives, are representative ofthe scope of the courseand are clearly stated.

Student evaluation: providesongoing and frequent diag-nostics to verify studentsreadiness for next modules;allows student to be contin-uously aware of his/herprogress in class and mas-

tery of content beyondgrades; provides clearexpectations and criteriafor assignments, includingrubrics for projects and

written assignments; speci-fies the relationship betweenindividual assignments andfinal course grade; providesdetailed, step-by-stepinstructions for each eva-luative exercise; providesguidelines for submittingassignment; provides avariety of feasible and con-tent relevant assignmentsor evaluative exercises;considers accommodationsfor ELL and SpecialEducation requirements.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

2. The course structureincludes adequate andappropriate methodsand procedures to assessstudents mastery ofcontent.

Valid and reliable assessmentsare frequent and timely andinclude online or proctoredtesting, performance assess-ments, projects, demonstra-tions, oral assessments, casestudies and portfolios; assess-ment and measurementstrategies include self-checkor practice types of assign-

ments to provide automated,immediate feedback to stu-dents, including content thatexplains the error or alludesto the content presentation;multi-layered approach topromote academic integrity;student participation in theevaluation of their work;and assignments resubmittedto demonstrate mastery.

1 2 3


11/16


3. Ongoing and frequentassessments are conductedto verify each students

readiness for the nextlesson.

Imbedded assessments andother assessment strategiesare in place throughout the

course to ensure accurateand up-to-date assessmentof student performance.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

4. Assessment strategies andtools make the studentcontinuously aware ofhis/her progress in classand mastery of the con-tent beyond letter grades.

Use of feedback tools andprocedures built into thecourse; students should beable to continuously self-monitor their academicprogress.

1 2 3

5. Assessment materialsprovide the teacher withthe flexibility to assessstudents in a variety of

ways.

Multiple versions of thesame test; test bank avail-able for the teachers use;alternative evaluation

methods available; test/question item banks orga-nized by the learning out-comes they are intendedto assess.

1 2 3

6. Grading rubrics andmodels of partially tofully completed assign-ments are provided tothe teacher.

Grading rubrics specify thecriteria for the assignments,and models of partially com-pleted assignments demon-strate the characteristics ofquality expected.

1 2 3

7. The grading policy andpractices are easy to

understand.

The grade values given toassessment and assignments

are balanced to reflect theirimportance in assessingmastery and the importanceof the content being assessed.

1 2 3


12/16

10


1. The course architecturepermits the online teacherto add content, activitiesand assessments to extendlearning opportunities.

Additions can be made with-in the course itself.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

2. The course accommo-dates multiple schoolcalendars; e.g., block,4X4 and traditionalschedules.

The course design allowsusers to adapt to variousschool calendars andschedules.

1 2 3

3. The course is easy tonavigate.

Consistent and predictablenavigation methods:Students can move logicallyand easily between areas ofthe course; color, graphicsand icons are used to guidethe student through thecourse; and a consistentlook and feel exist through-

out the course (consistenttext, colors, bullets, andheading styles).

1 2 3

4. Course makes maximumuse of the robust capabili-ties of the online mediumand makes these resourcesavailable by alternativemeans (video, CDs,podcasts).

Rich media are provided inmultiple formats for ease ofuse and accessibility.

1 2 3

5. Hardware, Web browserand software require-ments are specified.

Operating system, processorspeed, plug-ins and connec-tion speed are specified.

1 2 3

Section D: Technology

Course Architecture

User Interface

Technology Requirements and Interoperability

Accessibility

Technical Support

6. Prerequisite skills in theuse of technology areidentified.

Course describes what skillsand knowledge of Internetand software applications areneeded.

1 2 3

7. The course utilizes theappropriate contentspecific tools and soft-

ware.

Graphing calculators,probes, microscopes, wordprocessing, spreadsheets,presentation software, audiorecordings.

1 2 3


13/16


8. Interoperability technicalstandards allow sharingcontent among different

learning managementsystems.

Content is SCORM 1.2conformant at a minimum(includes IMS Content

Packaging 1.1.2); IMS QTIconformant assessments.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

9. Interoperability technicalstandards ensure sharingof questions, assessmentsand results with others.

Content is SCORM 1.2conformant at a minimum(includes IMS ContentPackaging 1.1.2); IMS QTIconformant assessments.

1 2 3

10. The course meets uni-versal design principles,Section 508 standardsand W3C guidelinesto ensure access for allstudents.

Content can be accessedby all students, includingstudents who use assistivetechnology to see, hearand move the information.

1 2 3

11. Online textbooks used ina course meet nationallyendorsed standards(NIMAS).

Electronic resources meetnational accessibility stan-dards for publishers toensure distribution of acces-sible, alternative versionsof textbooks and otherinstructional materials.

1 2 3

12. The course provideroffers the course teacherand school coordinatorassistance with technicaland course management.

The course provider offersproduct/course updates, ahelp desk, special assistanceand answers to frequentlyasked questions.

1 2 3

13. The course provideroffers orientationtraining.

The course provider providesadequate training to ensureuser understanding andknowledge to use eachcourse. This training shouldbe available when and in

what format the user needsit.

1 2 3


14/16

12


1. The results of peer reviewand student evaluationsof courses are available.

Meetscriteria

Partiallymeets

criteria


Notenough


1 2 3

2. Course provider usesmultiple ways of assessingcourse effectiveness.

Course is evaluated onmultiple criteria: studentcompletion rates; student

performance on in-courseassessments and on county,state, and/or national assess-ments or satisfaction surveys.

1 2 3

3. The course is evaluatedregularly for effectivenessand the findings used asa basis for improvement.

Course provider providesevidence of ongoing evalu-ations and associatedimprovements.

1 2 3

4. The course is updatedperiodically to ensuretimeliness.

This information may be ina separate publication fromthe course publisher. It alsocould be included in thecourse overview; e.g., course

updated June 2006.

1 2 3

5. The course provider isauthorized to operate inthe state in which thecourse is offered.

State review of courses andaccreditation of providersare ways to ensure that thecourse provider is of quality.

1 2 3

Section E: Course Evaluation and Management

Assessing Course Effectiveness

Updating Course

Accreditation

Data security

6. The teacher meets theprofessional teachingstandard established by astate licensing agency orthe teacher has academiccredentials in the field in

which he or she is teach-ing and has been trained

to teach online and touse the course.

See the SREB reportStandards for Quality OnlineTeachingfor additionalconsiderations.

1 2 3

7. Student informationremains confidential, asrequired by the FamilyEducational Rights andPrivacy Act (FERPA).

1 2 3


15/16


16/16

(06T06)

Documents

06T06 Checklist for Evaluating-Online-Courses