Michael Gage SOED 2016

WeBWorK within Moodle and WEPs

April 2016 Shape of Educational Data

Fairfax, VA

Michael Gage University of Rochester

https://hosted2.webwork.rochester.edu/gage/SOED_urls.pdf

WeBWorKWeBWorK

• The WeBWorK rendering engine, like STACK, does an excellent job of presenting mathematical homework questions and checking the students’ answers in a robust way — one which closely mimics paper-and-pencil homework.

• WeBWorK was originally developed with its own basic LearningManagementSystem (LMS).

• WeBWorK also has a webservice which allows it to plug into other LMS’s in order to share WW’s math smarts and the 30,000+ math questions (precalculus through linear algebra) that have been developed by the WW community.

WeBWorKOutline

The WW webservice can be used in several modes:

• 1. The Moodle quiz plugin which roughly mimics STACK.

• 2. The wwassignment plugin which supplies a homework assignment within Moodle. Provides single-sign-on and passes grades back to Moodle gradebook.

• 3. A recent LTI plugin, which may replace wwassignment, and allows plugins to Canvas, Blackboard, Moodle and other LMS that implement LTI 2.x with grade transfer. (w/ Geoff Goehle)

• 4. An HTML plugin which allows active WW problems within a webpage to be used as ungraded exercises. These also interoperate with MathBookXML — a write-once-print-in-all-formats authoring tool by Rob Beezer

WeBWorKMoodle quiz pluginhttps://devel3.webwork.rochester.edu/moodle

login:visitor password: visitor login:practice1 password: practice1 login:practice2 password: practice2 …

WeBWorKMoodle quiz pluginhttps://devel3.webwork.rochester.edu/moodle

login:visitor password: visitor login:practice1 password: practice1 login:practice2 password: practice2 …

WeBWorKMoodle quiz plugin

Moodle handles the presentation and quiz navigation. WeBWorK only renders the questions and evaluates the answers.

WeBWorK

Click “grade and finish”: the answer is graded and the correct answers are reported immediately.

WeBWorK

Problem 2. We’ll enter answers and click “submit attempt”.

WeBWorK

Some answers are wrong. I can resubmit and try again.

WeBWorK

I click “grade and finish”. My answers are recorded,

the correct answers are shown. and I can no longer change the answers.

WeBWorKMoodle quiz plugin

• The data collected by the Moodle quiz plugin is the same as the data collected by STACK

• The WW questions function similarly to STACK and you can mix STACK and WW questions in a quiz.

• The user interface appearance for the two question types is not yet identical — some more work could be done here.

• I could use some help from experienced Moodle users on this.

WeBWorKAssignment plugins

• The wwassignment and the LTI type are similar

• There is a script which allows a WeBWorK homework set of questions to be imported to the Moodle question bank.

• The linear algebra and ODE course on the next slide has homework sets in both formats.

WeBWorKLinear Algebra and ODE: Spring 2009

2009 Linear Algebra and ODE course demos both regular webwork homework AND Moodle quiz versions

WeBWorKStandard WW homework set

Single-sign-on (SSO) brings you to a standard webwork homework set (set 1) — grades are returned to Moodle

WeBWorKData collected by WW

1. Problem usage available in the library

2. Student progress and real time statistics

3. Past answers list (preliminary analysis by Roth, Record, et. al.)

4. “email the professor” conversations.

WeBWorKProblem usage statistics in OPL

• Usage: Number of times problem used

• Attempts: Average number of attempts for success

• Status: Success rate

WeBWorKStudent progress

• Final score on problem

• Number of attempts used

• There is also a percentage statistics page

Past Answer data

An answer

Hand analysis of past answer data

• Vicki Roth, V. Ivanchenko, and N. Record, “Evaluating student response to WeBWorK, a web-based homework delivery and grading system”, Computers & Education, vol. 50, no. 4, pp. 1462-1482, 2008.

• Results:• Analysis of answer data can distinguish between productive

progress and random guessing. • Guessing is extinguished for longer answers (or expressions).• Feedback from this study has been used to improve user

interface.• Analysis revealed an unexpected pattern: strong students

resubmitted wrong answers far more than other students

WW past answer data

WeBWorKCategorizing past answer data

• Student Response Model categories: 1. Reworking 2. Fixing an Entry Error 3. Re-submission (literal or mathematically equivalent) 4. Nonstrategic guessing 5. Nonsense

• https://math.webwork.rochester.edu/assessment/RaterTrainingManual.htm

• Can this analysis be automated and improved? BigData opportunity?

Email instructor data

Email instructor data

On Wed, Oct 15, 2014 at 7:04 PM, XXXXXXX <xxxxxx@u.rochester.edu> wrote:This  message was automatically generated by the WeBWorK system athttps://math.webwork.rochester.edu/webwork2/, in response to a request from

Click this link to see the page from which the user sent feedback:https://math.webwork.rochester.edu/webwork2/fall14mth141/6/24/?effectiveUser=xxxxx&showSolutions=1&showOldAnswers=1&displayMode=MathJax&showCorrectAnswers=0&showHints=1

***** The feedback message: *****

For Problem 24, Set 6, why is the answer +inf when the limit goes to -infof sqrt(x^2-8x+1)-x

I think I am forgetting an important rule...

***** Data about the problem processor: *****

Display Mode:         MathJaxShow Old Answers:     yesShow Correct Answers: noShow Hints:           yesShow Solutions:       yes

An “email the instructor” message

Email instructor data

***** Data about the user: *****

User ID:   XXXXXName:       Email:      Student ID: Status:     Enrolled ('C')Section:    Recitation:Comment:

***** Data about the problem: *****

Problem ID:                   24Source file:                  Library/Rochester/setLimitsRates3Infinite/ur_lr_3_13.pgValue:                        1Max attempts                  unlimitedRandom seed:                  3749Status:                       1Attempted:                    yesLast answer:        AnSwEr0001: -4        AnSwEr0002: INFNumber of correct attempts:   1Number of incorrect attempts: 0

Email instructor data

***** Data about the homework set: *****

Set ID:                    6Set header file:           screenSetHeader.pgHardcopy header file:      paperSetHeader.pgOpen date:                 09/29/2014 at 07:00am EDTDue date:                  10/10/2014 at 11:59pm EDTAnswer date:               10/14/2014 at 12:59am EDTVisible:                   yesAssignment type:           default

------------------------------------------------------------------------------------------------Instructor response:------------------------------------------------------------------------------------------------

Hi,

If you look conceptually at the question:sqrt(x^2-8x+1) when x is negative will be x^2 (positive) -8x (positive) so it will be constantly increasing and thus approaching infinity.

Let me know if that doesn't help and I will do my best to explain it better.Good luck tomorrow!

WeBWorKEmail instructor data

• Another opportunity for BigData analysis?

• Natural language processing?

WeBWorKAdvantages of interoperation

• Advantages of WW

• Many math questions in the OPL

• More than 770 institutions used it this semester.

• Potential for collecting data from many sources.

• Many authors contributing questions

• Advantages of Moodle

• Even more institutions use Moodle and adding WW in addition to STACK increases its math power.

WeBWorKInstitutions using WW

WeBWorKOPL with 33K questions

WeBWorKEmbedding WW in HTML

• https://hosted2.webwork.rochester.edu/gage/2016JMM/mth162_overview.html

<p><iframe width="800" height="400" src="https://hosted2.webwork.rochester.edu/webwork2/html2xml? &answersSubmitted=0& &sourceFilePath=Library/Union/setIntFTC/sc5_4_13.pg& &problemSeed=123567& &courseID=daemon_course& &userID=daemon& &course_password=daemon& &showSummary=1& &displayMode=MathJax& &problemIdentifierPrefix=102& &language=en& &outputformat=sticky"><br/></iframe></p>

Thank youhttps://hosted2.webwork.rochester.edu/gage/

2016WEPS.pdf for this presentation

LTI plugins and data collection are joint work with Goeff Goehle at Western Carolina University

WeBWorKCanvas LTI demonstration

• https://canvas.instructure.com/courses/1000526

• (You’ll need to sign up for an account.)

WeBWorKAdditional data available from WW

• The WeBWorK assignment mode collects additional data on the server for each problem.

• Past answer data

• Email the instructor data

• How can this data be analyzed effectively?

WeBWorK-BlackboardUses LTI standard. What is done here in Blackboard can in principal be done in

other LMS such as D2L or Canvas. It’s not as flexible as the connection to Moodle.

WeBWorK

Construct links with single sign on (SSO) to WeBWorK site or a Moodle site using LTI.

WeBWorK

Display a webwork course within the Blackboard page

WeBWorK

Display homework set for that week.

WeBWorK

Or display a single WeBWorK problem