Formative Assessment Materials for Large-Enrollment Physics Lecture Classes

Formative Assessment Materials for Large-Enrollment Physics Lecture Classes

David E. MeltzerDepartment of Physics

University of Washingtonwww.physicseducation.net

Supported by NSF DUE-0243258, DUE-0311450, PHY-0406724, and PHY-0604703

Real-time In-class Formative Assessment

• The Problem: How can the instructor assess students’ thinking during class and modify in-class instructional activities accordingly?

• Our Goal: Develop and test materials that both– provide a basis for in-class instructional activities, and– assist the instructor in monitoring student thinking,

moment-to-moment

in the context of large-enrollment classes

Our Materials: Carefully sequenced sets of multiple-choice questions

• Emphasize qualitative, conceptual items

• Make heavy use of multiple representations

• Designed to maximize student-instructor interaction in large classes

• Allow rapid assessment of student learning

• Assist instructors in structuring and guiding their presentations and instructional activities

Our Materials: Carefully sequenced sets of multiple-choice questions

• Emphasize qualitative, conceptual items

• Make heavy use of multiple representations

• Allow rapid assessment of student learning

• Assist in structuring and guiding the presentations and instructional activities

Motivation Research in physics education suggests that:

• Problem-solving activities with rapid feedback yield improved learning gains

• Eliciting and addressing common conceptual difficulties improves learning and retention

Active-Learning Pedagogy(“Interactive Engagement”)

• problem-solving activities during class time – student group work– frequent question-and-answer exchanges

• “guided-inquiry” methodology: guide students with leading questions, through structured series of research-based problems dress common learning

Goal: Guide students to “figure things out for themselves” as much as possibleuide students to “figure things out for themselves” as much as possible

Key Themes of Research-Based Instruction

• Emphasize qualitative, non-numerical questions to reduce unthoughtful “plug and chug.”

• Make extensive use of multiple representations to deepen understanding. (Graphs, diagrams, sketches, simulations, animations, etc.)

• Require students to explain their reasoning (verbally or in writing) to more clearly expose their thought processes.

Key Themes of Research-Based Instruction

• Emphasize qualitative, non-numerical questions to reduce unthoughtful “plug and chug.”

• Make extensive use of multiple representations to deepen understanding. (Graphs, diagrams, sketches, simulations, animations, etc.)

• Require students to explain their reasoning (verbally or in writing) to more clearly expose their thought processes.

Key Themes of Research-Based Instruction

• Emphasize qualitative, non-numerical questions to reduce unthoughtful “plug and chug.”

• Make extensive use of multiple representations to deepen understanding. (Graphs, diagrams, sketches, simulations, animations, etc.)

• Require students to explain their reasoning (verbally or in writing) to more clearly expose their thought processes.

Key Themes of Research-Based Instruction

• Emphasize qualitative, non-numerical questions to reduce unthoughtful “plug and chug.”

• Make extensive use of multiple representations to deepen understanding. (Graphs, diagrams, words, simulations, animations, etc.)

• Require students to explain their reasoning (verbally or in writing) to more clearly expose their thought processes.

Key Themes of Research-Based Instruction

• Emphasize qualitative, non-numerical questions to reduce unthoughtful “plug and chug.”

• Make extensive use of multiple representations to deepen understanding. (Graphs, diagrams, words, simulations, animations, etc.)

• Require students to explain their reasoning (verbally or in writing) to more clearly expose their thought processes.

Active Learning in Large Physics Classes

• De-emphasis of lecturing; Instead, ask students to respond to questions targeted at known difficulties.

• Use of classroom communication systems to obtain instantaneous feedback from entire class.

• Incorporate cooperative group work using both multiple-choice and free-response items

Goal: Transform large-class learning environment into “office” learning environment (i.e., instructor + one or two students)

Active Learning in Large Physics Classes

• De-emphasis of lecturing; Instead, ask students to respond to questions targeted at known difficulties.

• Use of classroom communication systems to obtain instantaneous feedback from entire class.

• Incorporate cooperative group work using both multiple-choice and free-response items

Goal: Transform large-class learning environment into “office” learning environment (i.e., instructor + one or two students)

“Fully Interactive” Physics LectureDEM and K. Manivannan, Am. J. Phys. 70, 639 (2002)

• Use structured sequences of multiple-choice questions, focused on specific concept: small conceptual “step size”

• Use student response system to obtain instantaneous responses from all students simultaneously (e.g., “flash cards”)

[a variant of Mazur’s “Peer Instruction”]

Results of Assessment

• Learning gains on qualitative problems are well above national norms for students in traditional courses.

• Performance on quantitative problems is comparable to (or slightly better than) that of students in traditional courses.

• Typical of other research-based instructional methods

Features of the Interactive Lecture

• High frequency of questioning

• Easy questions used to maintain flow

• Many question variants are possible

• Instructor must be prepared to use diverse questioning strategies

High frequency of questioning

• Time per question can be as little as 15 seconds, as much as several minutes.– similar to rhythm of one-on-one tutoring

• Maintain small conceptual “step size” between questions for high-precision feedback on student understanding.

High frequency of questioning

• Time per question can be as little as 15 seconds, as much as several minutes.– similar to rhythm of one-on-one tutoring

• Maintain small conceptual “step size” between questions for high-precision feedback on student understanding.

High frequency of questioning

• Time per question can be as little as 15 seconds, as much as several minutes– similar to rhythm of one-on-one tutoring

• Maintain small conceptual “step size” between questions for high-precision feedback on student understanding.

High frequency of questioning

• Time per question can be as little as 15 seconds, as much as several minutes– similar to rhythm of one-on-one tutoring

• Maintain small conceptual “step size” between questions for high-precision feedback on student understanding.

Curriculum Requirements for Fully Interactive Lecture

• Many question sequences employing multiple representations, covering full range of topics

• Free-response worksheets adaptable for use in lecture hall

• Text reference (“Lecture Notes”) with strong focus on conceptual and qualitative questions

Workbook for Introductory Physics (DEM and K. Manivannan, CD-ROM, 2002)

Curriculum Requirements for Fully Interactive Lecture

• Many question sequences employing multiple representations, covering full range of topics

• Free-response worksheets adaptable for use in lecture hall

• Text reference (“Lecture Notes”) with strong focus on conceptual and qualitative questions

Workbook for Introductory Physics (DEM and K. Manivannan, CD-ROM, 2002)

Curriculum Requirements for Fully Interactive Lecture

• Many question sequences employing multiple representations, covering full range of topics

• Free-response worksheets adaptable for use in lecture hall

• Text reference (“Lecture Notes”) with strong focus on conceptual and qualitative questions

Workbook for Introductory Physics (DEM and K. Manivannan, CD-ROM, 2002)

Curriculum Requirements for Fully Interactive Lecture

• Many question sequences employing multiple representations, covering full range of topics

• Free-response worksheets adaptable for use in lecture hall

• Text reference (“Lecture Notes”) with strong focus on conceptual and qualitative questions

Workbook for Introductory Physics (DEM and K. Manivannan, CD-ROM, 2002)

Curriculum Requirements for Fully Interactive Lecture

• Many question sequences employing multiple representations, covering full range of topics

• Free-response worksheets adaptable for use in lecture hall

• Text reference (“Lecture Notes”) with strong focus on conceptual and qualitative questions

Workbook for Introductory Physics (DEM and K. Manivannan, CD-ROM, Vol. II, preliminary edition, 2002)

Curriculum Requirements for Fully Interactive Lecture

• Many question sequences employing multiple representations, covering full range of topics

• Free-response worksheets adaptable for use in lecture hall

• Text reference (“Lecture Notes”) with strong focus on conceptual and qualitative questions

Workbook for Introductory Physics (DEM and K. Manivannan, CD-ROM, Vol. II, preliminary edition, 2002)

Further development supported by NSF under “Assessment of Student Achievement” program

Curriculum Requirements for Fully Interactive Lecture

• Many question sequences employing multiple representations, covering full range of topics

• Free-response worksheets adaptable for use in lecture hall

• Text reference (“Lecture Notes”) with strong focus on conceptual and qualitative questions

Workbook for Introductory Physics (DEM and K. Manivannan, CD-ROM, Vol. II, preliminary edition, 2002)

Further development supported by NSF under “Assessment of Student Achievement” program

Part 1: Table of Contents

Part 2: In-Class Questions and Worksheets, Chapters 1-8

Part 3: Lecture Notes Chapter 1: Electric Charges and Forces Chapter 2: Electric Fields Chapter 3: Electric Potential Energy Chapter 4: Electric Potential Chapter 5: Current and Resistance Chapter 6: Series Circuits Chapter 7: Electrical Power Chapter 8: Parallel Circuits Chapter 9: Magnetic Forces & Fields Chapter 10: Magnetic Induction Chapter 11: Electromagnetic Waves Chapter 12: Optics Chapter 13: Photons and Atomic Spectra Chapter 14: Nuclear Structure and Radioactivity

Part 4: Additional Worksheets Chapter 1: Experiments with Sticky Tape Chapter 2: Electric Fields Chapters 6 & 8: More Experiments with Electric Circuits Chapter 7: Electric Power, Energy Changes in Circuits Chapter 8: Circuits Worksheet Chapter 9: Investigating the Force on a Current-Carrying Wire Chapter 9: Magnetism Worksheet Chapter 9: Magnetic Force Chapter 9: Torque on a Current Loop in a Magnetic Field

Chapter 10: Magnetic Induction Activity Chapter 10: Magnetic Induction Worksheet Chapter 10: Motional EMF Worksheet Chapter 9-10: Homework on Magnetism Chapter 11: Electromagnetic Waves Worksheet Chapter 12: Optics Worksheet Chapter 13: Atomic Physics Worksheet Chapter 14: Nuclear Physics Worksheet

Part 5: Quizzes

Part 6: Exams and Answers

Part 7: Additional Material

Part 8: “How-to” Articles Promoting Interactivity in Lecture Classes Enhancing Active Learning The Fully Interactive Physics Lecture

Part 9: Flash-Card Masters

Part 10: Video of Class

AUTHORS: David E. Meltzer: Department of Physics and Astronomy, Iowa State University, Ames, IA 50011 dem@iastate.edu Kandiah Manivannan: Department of Physics, Astronomy, and Materials Science, Southwest Missouri State University, Springfield, MO 65804 kam319f@smsu.edu

video

Overall Tasks

• Develop question sequences for complete (two-semester) course

• Obtain feedback through in-class use to aid evaluation of assessment items

• Obtain interview data to validate items

• Do pre- and post-testing with standardized diagnostics to help monitor effectiveness

Design Strategies

• Not possible to pre-determine all possible discussion paths

• Knowledge of probable conceptual sticking points is important: guided by research

• Make use of standard question variants

Many question variants are possible

• Minor alterations to question can generate provocative change in context– add/subtract/change system elements (force,

resistance, etc.)

• Use standard questioning paradigms:– greater than, less than, equal to– increase, decrease, remain the same– left, right, up, down, in, out

Many question variants are possible

• Minor alterations to question can generate provocative change in context.– add/subtract/change system elements (force,

resistance, etc.)

• Use standard questioning paradigms:– greater than, less than, equal to– increase, decrease, remain the same– left, right, up, down, in, out

Many question variants are possible

• Minor alterations to question can generate provocative change in context.– add/subtract/change system elements (force,

resistance, etc.)

• Use standard questioning paradigms:– greater than, less than, equal to– increase, decrease, remain the same– left, right, up, down, in, out

Many question variants are possible

• Minor alterations to question can generate provocative change in context.– add/subtract/change system elements (force,

resistance, etc.)

• Use standard questioning paradigms:– greater than, less than, equal to– increase, decrease, remain the same– left, right, up, down, in, out

Many question variants are possible

• Minor alterations to question can generate provocative change in context.– add/subtract/change system elements (force,

resistance, etc.)

• Use standard questioning paradigms:– greater than, less than, equal to– increase, decrease, remain the same– left, right, up, down, in, out

Many question variants are possible

• Minor alterations to question can generate provocative change in context.– add/subtract/change system elements (force,

resistance, etc.)

• Use standard questioning paradigms:– greater than, less than, equal to– increase, decrease, remain the same– left, right, up, down, in, out

Many question variants are possible

• Minor alterations to question can generate provocative change in context.– add/subtract/change system elements (force,

resistance, etc.)

• Use standard questioning paradigms:– greater than, less than, equal to– increase, decrease, remain the same– left, right, up, down, in, out

Easy questions used to maintain flow

• Easy questions (> 90% correct responses) build confidence and encourage student participation.

• If discussion bogs down due to confusion, can jump start with easier questions.

• Goal is to maintain continuous and productive discussion with and among students.

Easy questions used to maintain flow

• Easy questions (> 90% correct responses) build confidence and encourage student participation.

• If discussion bogs down due to confusion, can jump start with easier questions.

• Goal is to maintain continuous and productive discussion with and among students.

Easy questions used to maintain flow

• Easy questions (> 90% correct responses) build confidence and encourage student participation.

• If discussion bogs down due to confusion, can jump start with easier questions.

• Goal is to maintain continuous and productive discussion with and among students.

Interactive Question Sequence

• Set of closely related questions addressing diverse aspects of single concept

• Progression from easy to hard questions

• Use multiple representations (diagrams, words, equations, graphs, etc.)

• Emphasis on qualitative, not quantitative questions, to reduce “equation-matching” behavior and promote deeper thinking

Interactive Question Sequence

• Set of closely related questions addressing diverse aspects of single concept

• Progression from easy to hard questions

• Use multiple representations (diagrams, words, equations, graphs, etc.)

• Emphasis on qualitative, not quantitative questions, to reduce “equation-matching” behavior and promote deeper thinking

Interactive Question Sequence

• Set of closely related questions addressing diverse aspects of single concept

• Progression from easy to hard questions

• Use multiple representations (diagrams, words, equations, graphs, etc.)

• Emphasis on qualitative, not quantitative questions, to reduce “equation-matching” behavior and promote deeper thinking

Interactive Question Sequence

• Set of closely related questions addressing diverse aspects of single concept

• Progression from easy to hard questions

• Use multiple representations (diagrams, words, equations, graphs, etc.)

• Emphasis on qualitative, not quantitative questions, to reduce “equation-matching” behavior and promote deeper thinking

Interactive Question Sequence

• Set of closely related questions addressing diverse aspects of single concept

• Progression from easy to hard questions

• Use multiple representations (diagrams, words, equations, graphs, etc.)

• Emphasis on qualitative, not quantitative questions, to reduce “equation-matching” behavior and promote deeper thinking

“Flash-Card” Questions

“Flash-Card” Questions

Examples of student response data(Algebra-based general physics at Iowa State University)

Intended to assist instructors at other institutions in selection and planning in use of item sequences

#1:

A: 0%B: 7%C: 93%D: 0%E: 0%

#2:

A: 10%B: 8%C: 77%D: 2%E: 5%

#7:

A: 2%B: 3%C: 3%D: 83%E: 9%

#8:

A: 0%B: 2%C: 8%D: 87%E: 3%

#9:

A: 0%B: 13%C: 7%D: 53%E: 22%

#10:

A: 67%B: 20%C: 9%D: 2%E: 0%

Problem “Dissection” Technique

• Decompose complicated problem into conceptual elements

• Work through problem step by step, with continual feedback from and interaction with the students

• May be applied to both qualitative and quantitative problems

Example: Electrostatic Forces

Problem “Dissection” Technique

• Decompose complicated problem into conceptual elements

• Work through problem step by step, with continual feedback from and interaction with the students

• May be applied to both qualitative and quantitative problems

Example: Electrostatic Forces

Problem “Dissection” Technique

• Decompose complicated problem into conceptual elements

• Work through problem step by step, with continual feedback from and interaction with the students

• May be applied to both qualitative and quantitative problems

Example: Electrostatic Forces

Problem “Dissection” Technique

• Decompose complicated problem into conceptual elements

• Work through problem step by step, with continual feedback from and interaction with the students

• May be applied to both qualitative and quantitative problems

Example: Electrostatic Forces

Problem “Dissection” Technique

• Decompose complicated problem into conceptual elements

• Work through problem step by step, with continual feedback from and interaction with the students

• May be applied to both qualitative and quantitative problems

Example: Electrostatic Forces

ff

Four charges are arranged on a rectangle as shown in Fig. 1. (q1 = q3 = +10.0 C and q2 = q4 = -15.0 C; a = 30 cm and b = 40 cm.) Find the magnitude and direction of the resultant electrostatic force on q1.

Question #1: How many forces (due to electrical interactions) are acting on charge q1?

(A) 0 (B) 1 (C) 2 (D) 3 (E) 4 (F) Not sure/don’t know

ff

Four charges are arranged on a rectangle as shown in Fig. 1. (q1 = q3 = +10.0 C and q2 = q4 = -15.0 C; a = 30 cm and b = 40 cm.) Find the magnitude and direction of the resultant electrostatic force on q1.

Question #1: How many forces (due to electrical interactions) are acting on charge q1?

(A) 0 (B) 1 (C) 2 (D) 3 (E) 4 (F) Not sure/don’t know

ff

Four charges are arranged on a rectangle as shown in Fig. 1. (q1 = q3 = +10.0 C and q2 = q4 = -15.0 C; a = 30 cm and b = 40 cm.) Find the magnitude and direction of the resultant electrostatic force on q1.

Question #1: How many forces (due to electrical interactions) are acting on charge q1?

(A) 0 (B) 1 (C) 2 (D) 3 (E) 4 (F) Not sure/don’t know

ff

Four charges are arranged on a rectangle as shown in Fig. 1. (q1 = q3 = +10.0 C and q2 = q4 = -15.0 C; a = 30 cm and b = 40 cm.) Find the magnitude and direction of the resultant electrostatic force on q1.

Question #1: How many forces (due to electrical interactions) are acting on charge q1?

(A) 0 (B) 1 (C) 2 (D) 3 (E) 4 (F) Not sure/don’t know

For questions #2-4 refer to Fig. 2 and pick a direction from the choices A, B, C, D, E, and F.

Question #2: Direction of force on q1 due to q2

Question #3: Direction of force on q1 due to q3

Question #4: Direction of force on q1 due to q4

For questions #2-4 refer to Fig. 2 and pick a direction from the choices A, B, C, D, E, and F.

Question #2: Direction of force on q1 due to q2

Question #3: Direction of force on q1 due to q3

Question #4: Direction of force on q1 due to q4

For questions #2-4 refer to Fig. 2 and pick a direction from the choices A, B, C, D, E, and F.

Question #2: Direction of force on q1 due to q2

Question #3: Direction of force on q1 due to q3

Question #4: Direction of force on q1 due to q4

For questions #2-4 refer to Fig. 2 and pick a direction from the choices A, B, C, D, E, and F.

Question #2: Direction of force on q1 due to q2

Question #3: Direction of force on q1 due to q3

Question #4: Direction of force on q1 due to q4

Let F2, F3, and F4 be the magnitudes of the force on q1 due to q2, due to q3, and due to q4 respectively.

Question #5. F2 is given by(A) kq1q2/a2

(B) kq1q2/b2

(C) kq1q2/(a2 + b2)(D) kq1q2/(a2 + b2)(E) None of the above(F) Not sure/Don’t know

Question #6. F3 is given by(A) kq1q3/a2

(B) kq1q3/b2

(C) kq1q3/(a2 + b2)(D) kq1q3/(a2 + b2)(E) None of the above(F) Not sure/Don’t know

Let F2, F3, and F4 be the magnitudes of the force on q1 due to q2, due to q3, and due to q4 respectively.

Question #5. F2 is given by(A) kq1q2/a2

(B) kq1q2/b2

(C) kq1q2/(a2 + b2)(D) kq1q2/(a2 + b2)(E) None of the above(F) Not sure/Don’t know

Question #6. F3 is given by(A) kq1q3/a2

(B) kq1q3/b2

(C) kq1q3/(a2 + b2)(D) kq1q3/(a2 + b2)(E) None of the above(F) Not sure/Don’t know

Let F2, F3, and F4 be the magnitudes of the force on q1 due to q2, due to q3, and due to q4 respectively.

Question #5. F2 is given by(A) kq1q2/a2

(B) kq1q2/b2

(C) kq1q2/(a2 + b2)(D) kq1q2/(a2 + b2)(E) None of the above(F) Not sure/Don’t know

Question #6. F3 is given by(A) kq1q3/a2

(B) kq1q3/b2

(C) kq1q3/(a2 + b2)(D) kq1q3/(a2 + b2)(E) None of the above(F) Not sure/Don’t know

Let F2, F3, and F4 be the magnitudes of the force on q1 due to q2, due to q3, and due to q4 respectively.

Question #5. F2 is given by(A) kq1q2/a2

(B) kq1q2/b2

(C) kq1q2/(a2 + b2)(D) kq1q2/(a2 + b2)(E) None of the above(F) Not sure/Don’t know

Question #6. F3 is given by(A) kq1q3/a2

(B) kq1q3/b2

(C) kq1q3/(a2 + b2)(D) kq1q3/(a2 + b2)(E) None of the above(F) Not sure/Don’t know (etc.)

Ongoing Curricular Development(Projects starting 2003)

• “Formative Assessment Materials for Large-Enrollment Physics Lecture Classes”

Funded through NSF’s “Assessment of Student Achievement” program

• “Active-Learning Curricular Materials for Fully Interactive Physics Lectures”

Funded through NSF’s “Course, Curriculum, and Laboratory Improvement – Adaptation and Implementation” program

Ongoing Curricular Development(Projects starting 2003)

• “Formative Assessment Materials for Large-Enrollment Physics Lecture Classes”

Funded through NSF’s “Assessment of Student Achievement” program

• “Active-Learning Curricular Materials for Fully Interactive Physics Lectures”

Funded through NSF’s “Course, Curriculum, and Laboratory Improvement – Adaptation and Implementation” program

Project Phases

• Complete the development of question sequences for Chaps. 10-14 of Volume II of Workbook for Introductory Physics

• Acquire baseline data by administering questions in class with electronic student response system

• Begin work on question sequences for initial chapters of Volume I of Workbook

Materials Development

• Carried out by DEM and Ngoc-Loan Nguyen (Iowa State U. graduate student)

• Created question sequences for electrodynamics, optics, modern physics, thermodynamics, and mechanical forces

• Acquired baseline data (at Iowa State University) for questions on electrostatics

• Currently carrying out editing and additional data acquisition

Contact:

David E. Meltzer

dem@physicseducation.net

www.physicseducation.net