Upload
susan-dickerson
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
A Joint MERLOT and European View of Multimedia in Physics
EPS Multimedia Working Group
MERLOT/Physics
MERLOT International Conference 2006Ottawa Canada, 8/10/2006
MIC 2006
Participants
EUPEN Working GroupM. Benedict (Hun), E. Debowska (Pol),B. Eckert (Ger), S. Geiner-Valkier (Neth),H.J. Jodl (Ger), L. Mathelitsch (Aus),I Ruddock (UK), E. Sassi (It), R. Sporken (Bel),
MERLOTT. Bradfield (OK), T. Colbert (GA), L. Keiner (SC), B. Mason (OK), T. Meldor (LA), J. Rauber (SD),S. Sen (NY), P. Sheldon (VA)
MIC 2006
EPS History
2002: EUPEN Working Group to Define Quality Standards. Example: Quantum Mechanics (MPTL-7, Parma)
2003: Apply Review Process. Example: Optics (MPTL-8, Prague)
2004: Classical Mechanics (MPTL-9, Graz) 2005: Thermal Physics (MPTL-10, Berlin) 2006: Electricity & Magnetism
(MPTL-11, Szeged)
MIC 2006
Goals & Methods
Present the high quality stuff, Recognize authors, Identify gaps
EPS –
• Work within one topic
• Collect many resources
• Select the best
• Apply criteria for full review of best
• Write up results
MERLOT –
• Search & submissions
• Rank materials for review
• Apply criteria for full review of best
• Review and publish
MIC 2006
Review Criteria (Rubrics)
EPS –
Motivation Content Method
MERLOT –
Quality of Content Effectiveness as a
Learning Tool Usability
MIC 2006
Review Criteria (Rubrics)
EPS – Motivation
User Friendly: Easy to start, understand, control, and is documented
Attractive: Appealing, interactive, and interesting
Clear Description and Context
Content Method
MERLOT – Quality Effectiveness Usability
Understandable: Runs easily
Intuitive: Attractive, controlled, input/output
Feedback: Clear communication
Documented
MIC 2006
Review Criteria (Rubrics)
EPS – Motivation Content
Relevant: Important topic and media use
Scope: Broad and “Profound” topic
Correct: Content accurate and models indicated
Method
MERLOT – Quality
Correct Models: Numerical, textual, notation
Important Topics: Standard or unique
Conceptual: Understand parameters
Effective Graphics Flexible: Multiple uses
Effectiveness Usability
MIC 2006
Review Criteria (Rubrics)
EPS – Motivation Content Method
Flexible: Broad audience and topics
Matching Target: Correct level, background, objectives
Realization: Media is well used
Documentation: Operation, references, teaching process
MERLOT – Quality Effectiveness
Learner: Level, challenge, control
Relevant Knowledge: Learning goals, application
Experience: Dynamic, flexible, interactive, progress
Feedback: Clear, immediate, positive
Usability
MIC 2006
Review Scales
EPS – +2 to -2 Indicated for each
sub-question Averaged
MERLOT – 1 – Major Problems 2 – Works, but has
errors 3 – Works well,
useful 4 – Excellent, notable
features 5 – Outstanding and
unique
MIC 2006
Agreement?
MERLOT Reviewers: Average Absolute Value of Difference between Reviewers (Scale of 1 – 5, Reviews 3 – 5, 135 items)
Quality: 0.56 (3.8) Effectiveness: 0.60 (3.65) Usability: 0.69 (3.9) Total: 0.57 (3.72)
MIC 2006
Agreement?
MERLOT Reviewers: Average Difference between Reviewers, Time Dependence
Quality: 0.57 (00 – 03), 0.55 (04 – 06) Effectiveness: 0.7 (00 – 03), 0.5 (04 – 06) Usability: 0.7 (00 – 03), 0.69 (04 – 06) Total: 0.6 (00 – 03), 0.53 (04 – 06)
MIC 2006
Agreement?
MERLOT Reviewers: Average Difference between Reviewers, Score Dependence
(N5 = 20, N4 = 61, N3 = 39)
5 4 3
Qual 0.55 0.53 0.55
Eff 0.35 0.62 0.64
Use 0.45 0.66 0.82
Tot 0.35 0.62 0.52
MIC 2006
2002 Quantum Mech
EPS: ~30 items, 7 recommended MERLOT:
2 award winners, 5, 4 (2), 2 not reviewed = 0 (1 in German
Joffre QM (2003 MERLOT Classic) de Raedt QM (No MERLOT Review)
General Recommendations Results from Workshop
MIC 2006
2003 Optics
EPS: ~250 items, 5 Recommended
MERLOT: 5, “4.5” (2), 0 (2)
Paul Falstad Math & Physics Applets
Report & Recommendations
MIC 2006
2004 Mechanics, Different Lists
EPS: ~250 items 7 Recommended Merlot: 5 (2), “4.5” (2), 4, 3, 0 (German)
MERLOT: ~700 items 4 Other Recommendations EPS: Rec (not listed), High quality, 0 (2)
Chaos Collection
Report & Recommendations
MIC 2006
2005 Thermal Physics, Joint
~150 items, 55 Reviewed 1 Highly Recommended EPS & MERLOT
(Engineering) 7 Recommended 1 Disagreement
Expert System for Thermodynamics Stat Mech in Intro Physics
Report & Recommendations
MIC 2006
2006 Elect & Magn, Joint
950 items, 200 Reviewed 6 Highly Recommended EPS & MERLOT 6 Recommended by both 5 Disagreements (Some may not have been
reviewed)
MIT “TEAL” (OCW) Physlet Tutorial Pages
MIC 2006
Comments & Conclusions
Good agreement w/ independent reviews Quality rating for larger collections
Many topics repeated Many Simulations, not video experiments
Instructional aspects rare How to put into a class rarer