Upload
hilary-mcdaniel
View
230
Download
0
Tags:
Embed Size (px)
Citation preview
MathPASS: A Remedial Mathematics System with MathPASS: A Remedial Mathematics System with Automated Answer CheckingAutomated Answer Checking
Wei Su1 Paul S.Wang2 Lian Li1 Department of Computer Science, Lanzhou University, PRC Department of Computer Science, Kent State University, USA [email protected]
July, 2010 on CICM’10
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
OutlineOutline
MathPASS Overview
Question Prototype and MAML
Answer Inputting and MathEdit
Answer Checking and MACP
User Statistics
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
1. Overview1. Overview
MathPASS is a Web-based intelligent mathematics assessment system
A drill-and-practice system which has been applied in remedial mathematics courses at KSU.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Taking Assignment Page in MathPASSTaking Assignment Page in MathPASS
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
System Modules of MathPASSSystem Modules of MathPASS
User Management
User Management
core
StudentManagement
StudentManagement
Account Setting
Account Setting
AssignmentManagement
AssignmentManagement
Class Management
Class Management
QuestionManagement
QuestionManagement
Practice Practice
GradeManagement
GradeManagement
QuestionsGeneration
QuestionsGeneration
AnswerChecking
AnswerCheckingAnswer
Inputting
AnswerInputting
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
The System Architecture of MathPASSThe System Architecture of MathPASS
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
2. Question Generation2. Question Generation
The questions in MathPASS could be generated automatically by question prototypes on each time when students take the assignment.
A question prototype is a mathematical question template for a specific question pattern containing random parameters satisfying well-defined mathematical conditions.
More than 400 question prototypes have been created in MathPASS by the combined efforts of mathematics teachers and mathematics education experts.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Question Prototypes and InstancesQuestion Prototypes and Instances
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
• The table also gives the statistic result including taken times, correct times, and the correct ratio of the question. • All of the statistic data can be accessed by a MathPASS teacher user who may use these data in preparing his/her course.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
MAML-Mathematical Assessment Markup LanguageMAML-Mathematical Assessment Markup Language
Paper (homework, quiz, examination) elements<paper> <pdescription> <ptitle> <psubject> <pstarttime> <pendtime> <psite> <pdifficulty> <pscore> <pspecification> <pschool> <pgrade> <pclass> <pname> <pstudentid> <psection>
Question elements<questionset> <question> <qdescription> <qbody> <qchoice> <qtext> <qblank> <qmathml> <qsvg> <qimage> <qtocorrect> <qparameter> <qkey>
Question Prototype elements < prototype > < prototypeBody> <prototypeCond> <require> <requirenot><and><or> <test><apply>
Description elements <qtopic> <qcorrectness> <qdifficulty> <qrscore> <qanswer> <qscore>
Presentation elements<qrow> <qbr>
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
An example of MAML for Question PrototypeAn example of MAML for Question Prototype<prototype>
<prototypeBody> <apply><or><plus/><minus/></or></apply> <apply name=“subexp1”> <divide/> <cn type=“int” valtype=“rand” minVal=“1” maxVal=“9”>A</cn> <cn type=“int” valtype=“rand” minVal=“1” maxVal=“15”>B</cn> </apply> <apply name=“subexp2”> <divide/> <cn type=“int” valtype=“rand” minVal=“1” maxVal=“9”>C</cn> <cn type=“int” valtype=“rand” minVal=“1” maxVal=“15”>B</cn> </apply></prototypeBody>
<prototypeCond> <require> <test value=“subexp1” compare=“eq”>SF</test> <test value=“subexp2” compare=“eq”>SF</test> </require> </prototypeCond>
</prototype>
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
3. Answer Inputting and MathEdit3. Answer Inputting and MathEdit
An intuitive and convenient mathematical expression input tool is very important for a mathematical CAA system to enter answer.
MathEdit, a Web-based visual interactive mathematical expression editor, aims to provide an integrated solution of on-line entering and editing of mathematical expressions.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
MathEdit Overview MathEdit Overview
MathEdit provides convenient and intuitive graphical user interface.
MathML, OpenMath, LaTex, Infix, Picture (Gif, Png), PDF could be generated by MathEdit.
It also provides well-defined API.
It can work with IE and the Mozilla-family browsers
Written in JavaScript and MathML.
http://mathedit.lzu.edu.cn or http://wme.cs.kent.edu/mathedit
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
MathEdit Authoring EnvironmentMathEdit Authoring Environment
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
The architecture of MathEditThe architecture of MathEdit
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Some Examples of Math expressionSome Examples of Math expression
Demo
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
MathEdit Infix VersionMathEdit Infix Version
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
MathEdit Usage ExamplesMathEdit Usage Examples
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
4. Answer Checking and MACP4. Answer Checking and MACP
Overview of MACP
Data Format of MACP
Answer Grade Rule
Full Simplification Form
Special Syntax Form
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
4. 1 Overview of MACP4. 1 Overview of MACP
Mathematical Answer Checking Protocol (MACP) is an access protocol for communication between MACP service and its client.
MACP service is a Web service for checking answers and grading questions to mathematics of middle school and college.
The MACP service aims to grade the user answer through
– verifying expressions equivalence
– checking the expression form of the answer
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Overview of MACPOverview of MACP
The standard answer could be provided by the Computer-aided Assessment (CAA) system which is using MACP or automatically generated and computed by the MACP service.
The implementation of MACP is based on Representational State Transfer (REST).
The request data and service response will be encapsulated into JSON code.
In internal, MACP service use computer algebra systems (Maxima) to verify the expression equivalence and compute the correct answer.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
A usage example of MACPA usage example of MACP
MathEdit
AnswerGrade
Program
CAS Question
Generator
CAAPHP Proxy
MACP Service Web Server
MACP Client
BrowserAJAX
POST
JSON String
MACP Request
MACP Response
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
4.2 DATA FORMAT OF MACP4.2 DATA FORMAT OF MACP[{"name":"<check item name>",
"cmd":"<check command>",
"response":"<response list>",
"exprs":[{
"expr":"<expression name>",
"encoding":"<encoding format>",
"role":"<expression role>",
"value":"<expression itself>",
"form":"<required expression form>",
"op":"<question operation>",
"var":"<variables>"
},...]
},...]
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
The Response DataThe Response Data[{
"name":"<check item name>",
"status":"<status>",
"error":"<error code>",
"correctness":"<correctness>",
"response":[{
"rname":"<response item name>",
"value":"<response item value>"
},...]
},...]
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
ExampleExample2x2x22 + 8x + 7 = 0 + 8x + 7 = 0
[{"name":"c1",
"cmd":"(u1=s1&u2=s2)|(u1=s2&u2=s1)",
"request":["ranme":"u1.mmlp","rname":"u2.mmlp"],
"exprs":[
{ "expr":"s1","encoding":"mmlc","role":"stdAnswer",
"value":"<math><plus/><apply><minus/><cn>2</cn></apply><apply><divide/>
<apply><root/><cn>2</cn></apply><cn >2</cn></apply></math>"},
{ "expr":"s2","encoding":"mmlc","role":"stdAnswer",
"value":"<math><minus/><apply><minus/><cn>2</cn></apply><apply>
<divide/><apply><root/><cn>2</cn></apply><cn >2</cn></apply></math>"},
{ "expr":"u1","encoding":"infix","role":"usrAnswer","value":"(-4-sqrt(2))/2"},
{ "expr":"u2","encoding":"infix","role":"usrAnswer","value":"sqrt(8)/2-2"}
]
}]
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Response Data of Response Data of the Examplethe Example
[{
"name":"c1",
"status":"normal",
"correctness":"96%",
"details":"<u2> is not simplest",
"response":[
{ "rname":"u1.mmlp",
"value":"<math><mfrac><mrow><mo>-</mo><mn>4</mn><mo>- </mo>
<msqrt><mn>2</mn></msqrt></mrow><mn>2</mn></mfrac></math>"},
{ "rname":"u2.mmlp",
"value":"<math><mfrac><msqrt><mn>8</mn></msqrt>
<mn>2</mn></mfrac><mo>-</mo><mn>2</mn></math>"
}]
}]
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
4.3 Answer Grade Rule4.3 Answer Grade Rule
When can we say an answer is ”right”? – The answer which student entered can be computed with
the standard answer by CAS. Obviously the answer is wrong if the computation result is not equivalence in algebra. However we cannot easily say the answer is right even though they are equivalence.
8 5
5 1
6 y x
1/2
5 8
25 1
6 x y5 5 3
1 5
6y x y8 5
1 1 1*5
6 y x
5
8
5
6
x
y
2 3 3 5
1 5
6x y x y
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Answer Grade RuleAnswer Grade Rule
Hundreds of emails from students to report that the answer they entered is correct but the system grade it wrong.
On the other hand, some mathematical teachers also report that some student answers should be treated as ”wrong” or ”partial right” while the system grade them right.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
ClassificationClassification
Most correct answer or best answer be classified into two broad categories:
– simplest form
– special syntax form
This taxonomy does not separate strictly all the answers into two totally different categories. One part of the answer with special syntax form may also be required as a simplest form.
However this taxonomy has been found to be useful for evaluating the answer of students and solving the issues of partial credit.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
4.4 Full Simplification Form4.4 Full Simplification Form
The question of this category included:
– simplification,
– calculation,
– writing the equation of a function,
– writing the domain of expression,
– …
Exmaples:
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Rule-based Grade MethodRule-based Grade Method
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Canonical FormCanonical Form
Rewrite Rule 1 (Remove Redundant Parentheses)
Rewrite Rule 2 (Commutative Rule of Addition andMultiplication)
Rewrite Rule 3 (Multiplication of Fractions)
Rewrite Rule 4 (Representation of Radical)
The expression will not be simplified during the canonical process and the length and complexity of the expression were not changed during the rewriting.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Simplest forms of a non-symbolic constant Simplest forms of a non-symbolic constant expressionexpression
Simplest Natural Number (SN)
Simplest Integer (SI)
Simplest Fraction (SF)
Simplest Decimal Number (SD)
Simplest Exponent/Radical (SE)
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Checking Rule AChecking Rule A
A non-symbolic constant expression is in simplest form when it satisfies one of the following conditions:
– It is one of the SN, SI, SD, SF, SE;
– If a non-symbolic constant expression is the addition or minus of one SE number and one number of SN, SI, SD, SF, the expression is a simplest form expression.
– If a non-symbolic constant expression is the multiplication of one SE number and one number of SN, SI, SD, or SF, where the base of the SE number is different with the other number, the expression is a simplest form expression.
– If a non-symbolic constant expression is the dividing of one SE number and one number of SN, SI, or SD, where the base of the SE number is different with the other number, the expression is a simplest form expression.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
TermTerm
In elementary mathematics, a term is either a single SP, or the product of several SPs separated from another term by a + or - sign in an overall expression. For example, in 3*9+4*x+5*y*z*w, the 3*9, 4*x, and 5*y*z*w are all terms.
A term can be written in form
C*S*V
where C is a non-symbolic expression, S is a symbolic expression and V is a variable expression.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Checking Rule BChecking Rule B
A term of an expression is in simplest form when it satisfies the following conditions:
– each variable or symbolic constant only occurs once;
– the non-symbolic part is in simplest form.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Checking Rule CChecking Rule C
A polynomial expression is in simplest form when it satisfies the following conditions:
– each term of the polynomial expression is in simplest form;
– the variable parts are different for each two terms of the polynomial.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
4.5 Special Syntax Form4.5 Special Syntax Form
Students need write the answer expression as an appointed syntax form such as multiplication of polynomials, a sum or difference of logarithms.
This kind of answer does not need to be reduced to lowest terms. Contrarily, the question may be lowest terms and obviously the answer should not be same with the question.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
ExamplesExamples
The questions of this category include
– factoring,
– solving equation,
– rewriting the expression as request,
– find the function as request,
– …
Example: Factor Completely, leaving only positive exponents in your answer
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Special Syntax FormSpecial Syntax Form
Factoring Polynomial (FP)
Normalized Scientific Notation (NSN)
Sum and Difference of Logarithm (SDL)
Positive Exponent (PE)
…
The user may use logical combination of these special form as a request form. For example a client may send SF|SI to statement the answer could be a simplest fraction or a simplest integer.
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
Pattern Language for Mathematical ExpressionPattern Language for Mathematical Expression
Mathematical Pattern Language provides a concise and flexible means to represent classes of expressions.
A simple example of a mathematical pattern is the expression 2+x^$. This mathematical pattern represents the class of expressions with the form 2+x^[any exponent].
– Regular Expressions--> String
– Mathematical Pattern Language--> Mathematical expression
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
An Example of Mathematical Pattern LanguageAn Example of Mathematical Pattern Language
sin$1v+cos$1v <mq:query> <plus/> <apply> <sin/> <mq:st valuetype=“var” class=“1”> </mq:st> </apply> <apply> <cos/> <mq:st valuetype=“var” class=“1”> </mq:st> </apply> </mq:query>
sinx+cosx
sina+cosa
sin2+cos2
sinx+cosy
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
5. Statistics5. Statistics for MathPASSfor MathPASS
Name Numbers Name Numbers
Teachers 103 Students 6204
Courses 547 Assignments 3731
Taken Assignments 121770
Taken Questions 1282095
Wei Su
http://mathpass.math.kent.edu
Lanzhou University
6. Future Work6. Future Work
Improve the MACP and answer grade rule.
Add the geometry question in MathPASS.