Physics Laboratory Performance Assessment at UNC-CH

Contributed talk BG02 presented Sunday, January 4, 2015

at the Winter AAPT meeting, San Diego, CA

Duane L. DeardorffDir. of Undergraduate Laboratories in Physics and Astronomy at

The University of North Carolina at Chapel Hill

www.physics.unc.edu/~deardorfwww.physics.unc.edu/labs

Overview• For the past 13 years, an individual, hands-on lab exam

has been administered near the end of our introductory physics laboratory courses:– Phys24 & 25 (algebra-based for bio, pre-med majors)

Combined: N 500 students per semester – Phys26 & 27 (calc-based for physics, comp sci, etc.)

Combined: N 200 students per semester

• Labs are mostly traditional, experimental verification.• Lab exam counts for 20% of total lab score, which is

25% of course grade. This is roughly equivalent to 2 lab report scores since there are 9 labs/course.

• A sample lab exam for each course (with answers) is on lab website so students know what to expect and can practice with equipment in Tutorial Center.

Why have a lab exam?• Lab reports generally do not assess lab skills.• Individual lab practicum provides incentive for all

students to learn lab skills (not just hands-on learners).• Help ensure that TAs teach proper methodology.• To discover weak areas of curriculum and TA training

that need improvement.• Can be used as a proficiency exam for students trying to

test out of lab.• To improve discrimination of lab scores.

(Some faculty complain that lab scores are too high, have narrow distribution, and are not consistent with course exam scores.)

Lab exam learning objectives· Correctly use lab equipment to measure physical

quantities.· Correctly report each measurement with label, accurate

value, reasonable uncertainty, sig. figs., and proper unit.· Analyze experimental results and write a valid conclusion.· Identify and explain choice of lab procedures used.· Apply familiar experimental techniques to new situations.· Determine final result and uncertainty from given data.· Identify the primary source of error in an experiment.· Properly construct graphs and interpret results.· Design a simple experiment to measure a desired quantity.

Format• Students work individually on written and hands-on

exercises that are similar to tasks they (should) have already done in earlier labs.

• Open-book, open notes: students are allowed to use any resources except other people.

• The exams are designed to require about 1 hour, but students are allowed 2 hours (normal lab period)

• Sample lab exams with answers are provided on lab website. Equipment is available in Physics Tutorial Center for students to practice and get help.

• Lab exams are modified only slightly from semester to semester (usually only 1 or 2 questions are changed.)

• Students do not get to keep lab exam papers.

Instructions to StudentsInstructions: Work individually to complete each exercise to the

best of your ability, show all your work, and clearly explain your answers in the spaces provided or on the back of these papers.

Be sure to record all measurements (in SI units) and show all calculations. For items that require a numerical result, write your answer as you would for a formal lab report, including a meaningful label to identify a value. Your answer will be graded based on the accuracy of your result and proper reporting of uncertainty, significant figures, and units.

Once the lab exam begins, you are not permitted to receive any assistance from your TA or other students. However, you may use your lab manual, graded lab reports, notes, and textbook as resources for this exam. The questions may be answered in any order, so adjust your work according to the availability of the lab equipment.

Honor Pledge: All work presented here is my own. _________

Lab exam content - Mechanics

1. Use photogates to measure accel. of glider on air track

2. Measure density of nickel coins. Are they pure nickel?

3. Given time and distance data, find acceleration and xo.

4. Use Vernier caliper to find radius of a steel ball

5. Report best estimate of length of hall from 6 measurements

6. Determine mass of rotating object from graphical results

7. Calculate error from flag on air track glider rotated 20°

8. Accuracy and precision of a simple pendulum

9. How to report sin(80° ± 1°)?

Ranked roughly according to question weighting:

Lab exam content – E&M

1. Connect circuit shown and measure currents with DMM.

2. Use oscilloscope to measure time constant for RC circuit.

3. Given a trace, find Vpp, Vrms, and frequency of signal.

4. Calculate RT for 2 resistors (with 5% tolerance) in parallel

5. Find n for block shown with refracted ray passing through.

6. Given a converging lens, estimate its focal length.

7. Given radioactive decay data, find half-life.

8. Sketch graph of I vs V for light bulb that is non-ohmic.

9. Given ammeter reading, report current value and uncert.

Ranked roughly according to question weighting:

Grading

A detailed rubric was developed on Excel for scoring by hand or by computer– Grading takes ~5 min/student – Objective grading except for explanations of procedures– Measured values are generally considered accurate if they

are within ± 2U of the median expert value, where U is the combined standard uncertainty (RSS of Type A and Type B components).

– Uncertainty values are generally acceptable up to 5*U– Good reliability (avg. difference between graders is 2%)

Grading – Target Weights

Component Weight

Accurate values 40%

Units 15%

Uncertainty 15%

Significant figures 10%

Labels 5%

Procedures 10%

Explanations 5%

Total = 100%

Grading – Typical Results

Component Weight Avg. Score

Accurate values 40% 54%

Units 15% 79%

Uncertainty 15% 35%

Significant figures 10% 48%

Labels 5% 67%

Procedures 10% 85%

Explanations 5% 66%

Historical Physics Lab Exam Averages

0102030405060708090

100

Fall 2001 Spring2002

Fall 2002 Spring2003

Fall 2003 Spring2004

P24

P25

P26

P27

Spring 2011 Fall 2011 Spring 2012 Fall 2012 Spring 2013 Fall 20130

10

20

30

40

50

60

70

80

90

Historical Physics Lab Exam Averages

P104P105P116P117

Historical Trends• Average scores vary from semester to semester, but

are mostly constant over time: ~65% (similar to course exam scores)

• Spread of scores (SD) each semester is consistent.~15%

• Slight decline in exam performance in recent years, not sure why. Apparently students are not cheating much from old exams!

• Exam is robust even without significant changes from year to year.

Typical results (Physics 105, Spring 2012)

Conclusion: Scores do not increase or decrease throughout the week.

Correlation analysis for Phys24

Lab exam scores vs: R

Lab report scores 0.3 - 0.4

Course exam average 0.5 - 0.6

WebAssign homework 0.4 - 0.5

Class participation 0.2 - 0.3

Conclusion: Lab exam does not duplicate lab report scoring, but it does correlate well with course exams. (Correlation between exam scores is ~0.5)

Conclusions• The lab exam provides a rich set of research data.• Lab exam and lab reports are different assessments – together

they give a more valid measure of a student’s achievement.• 90% of students responded that they felt the exam was fair -

evidence of good content validity• Motivates students (and TAs) to learn lab skills• Opportunity to practice is important for student learning• Faculty are generally pleased with lab exam and scores that are

more consistent with course performance • Student performance is not nearly as good as we would like for

our students to achieve. We still have much work to do! • Not surprisingly, performance depends on training and

experience in class.

Links to Sample Documents

• Statement from lab manual and website• Sample lab exams with answers• Phys24/26 Lab Exam, Spring 2004• Grading rubric• Phys25 Lab Exam, Spring 2004• Grading rubric• Instructions for TAs