Ted J. Branoff, Ph.D. North Carolina State University College of Education Department of...

Ted J. Branoff, Ph.D.North Carolina State UniversityCollege of EducationDepartment of Mathematics, Science and Technology Education

Large Course Redesign of GC120Moving the Foundations of Graphics Course from a Face-to-face to a Blended Learning Format

GC120 - Content

Previous Structure of GC120

• Daytime sections taught by full-time faculty– Met 2 times per week for 110 minutes

– 2-3 sections rotated into a computer lab during these meetings

• Night sections taught by part-time faculty– Met once per week for 3.5 hours

– Typically had their own dedicated computer lab

Issues with GC120

• Consistency between sections– SolidWorks instruction– Evaluating SolidWorks assignments– Proportion of sketching and SolidWorks assignments

• Access to SolidWorks help outside of class• Large number of students on wait list (about 130

each semeter)

Large Course Redesign Grant

• Summer 2008 - Submitted proposal for DELTA Large Course Redesign Grant.

• Proposed moving all sections of GC120 to a hybrid or blended format.

• Proposed more laptop sections.• Proposed help sessions in Elluminate.• Proposed an automated SolidWorks grader.

Moving Sections to Blended Format

• During the Fall of 2008 content was moved from WebCT to Moodle.

• Currently all day-time sections of GC120 are blended.

• Continue to train night faculty.

New Course Structure• Daytime sections meet in class once per week

– Answer students’ questions– Talk about key concepts for the week– Check homework

• Online content completed outside of class– Voiced-Over PowerPoint Presentations– Voiced-Over SolidWorks Demonstrations– Videos of an instructor sketching– Study Guides– Low-stake Online Assessments in Moodle – 10-20 questions – 2

Attempts

More Laptop Sections

• Gradually moving to more laptop sections.

• Have renovated Poe 106 to handle 60 students.

• Still have many issues to overcome with student owned computers.

Online Help Sessions

• Offered some help Elluminate help sessions during the Fall 2008 semester for exam reviews and project help.

• Students still seem to prefer face-to-face help sessions.

• We have started offering face-to-face help sessions each semester.

Automated SolidWorks Grader

• Began programming in Fall 2008.

• Made some improvements during Fall 2009 with departmental funding.

• Still have issues with ease of use for faculty.

Fall 2008 Survey

• Prior to moving to Moodle, compared 3 pilot sections of blended structure to F2F sections.

• Materials were organized in open web pages.• Conducted in the 13th week of semester.• Questions about:

– Previous experience with online or hybrid courses– Instructional preference– General order when navigating through online materials

Fall 2008 Results

• Some previous experience with online and blended courses.

• Prefer blended over face-to-face.

• Prefer face-to-face over online.

• Students navigated through material in many different ways.

Fall 2008 Results

• No differences between face-to-face and blended sections on the midterm exam.

• Significant difference between the groups for the final exam.

Moving to Moodle

• We can now look at how students are actually navigating through materials.

• What are the best strategies for success?

Fall 2009 Study Research Questions• In what order did students progress through the

materials related to the textbook?

• What was the typical number of attempts at each assessment?

• Did students who attempted all of the online assessments perform better on the midterm and final exams than students who only attempted a few assessments?

Participants

GC120 Section

Frequency %

003 55 34%

004 56 34%

005 53 32%

Total 164 100%

Participants – Academic Year

Year Frequency %

Freshmen 4 2%

Sophomores 115 70%

Juniors 29 18%

Seniors 16 10%

Total 164 100%

Participants – Academic MajorAcademic Major Frequency %

Aerospace Engineering 17 10%

Civil Engineering/Construction Man. 45 27%

Mechanical Engineering 44 27%

Other Engineering Majors 29 18%

Education 7 4%

First Year College 8 5%

Other Majors 14 9%

Total 164 100%

Moodle LMS

Moodle Lesson Format

Voiced-Over Presentations

Sketching Videos

Results

In what order did students progress through the materials related to the textbook?

Results – Order of Navigation

Results

What was the typical number of attempts at each assessment?

Results – Number of Assessment Attempts

Results

Did students who attempted all of the online assessments perform better on the midterm and final exams than students who only attempted a few assessments?

Results – Midterm Means by Assessment Attempts

Results – Exam Means by Assessment Attempts

Analysis – Correlation between Midterm Examand Online Assessment Attempts

Spearman’s RhoAssessment

Attempts Before Midterm

Midterm Exam

Assessment Attempts Before the Midterm Exam

Correlation Coefficient

1.000

Sig. (2-tailed) .

N 164

Midterm Exam

Correlation Coefficient

.233** 1.000

Sig. (2-tailed) .004 .

N 164 164

** Correlation is significant at the 0.01 level (2-tailed)

Analysis – Correlation between Final Examand Online Assessment Attempts

Spearman’s RhoAssessment

Attempts Before Final Exam

Final Exam

Assessment Attempts Before the Final Exam

Correlation Coefficient

1.000

Sig. (2-tailed) .

N 164

Final Exam

Correlation Coefficient

.283** 1.000

Sig. (2-tailed) .000 .

N 164 164

** Correlation is significant at the 0.01 level (2-tailed)

Discussion / Conclusions• While a number of students followed the

explicitly recommended order of material use (i.e., view the streaming video before attempting the quiz), many students took alternate approaches to usage order.

• In addition, this usage order also changed over the course of the semester.

Discussion / Conclusions• The weekly online quiz assessments were only

worth a total of 10% of students’ final grade and the midterm and final worth collectively 40% of the grade.

• It is our assumption that students primarily saw the value in the weekly assessments as preparing them for the larger summative assessments of the midterm and final.

Discussion / Conclusions• The data collected seem to support the

conclusion that those students who attempted more weekly assessments (and/or made use of the streaming videos) did better on the midterm and final exams.

Discussion / Conclusions• Probably the most important finding of this study is

that the logging tools provided in Moodle provides a powerful tool for instructors to gather and analyze data on how students make use of the resources provided online.

• It is now easier to chart student trends and performance in a more accurate way than in past incarnations of this course.

Discussion / Conclusions• This provides instructors with better information to

use in the redesign of course materials for the future.

• Since the production of the multimedia learning resources is a labor-intensive practice, this formative data provides valuable evidence as to whether such material is being used by students and whether it provides real educational value.

Future Research• Develop methods for more fine-grained analysis of log

data.– More data points over the semester to better understand

trends– Analysis of first versus second tries on quizzes– Use of SCORM-compliant learning resources in conjunction

with Moodle that allow richer data collection usage (e.g., how long did they view a video and how many times did they stop and start it)

Questions?