IMMEX: Providing Insight into Problem Solving Using Technology Charlie Cox Preliminary Oral Defense

IMMEX: Providing Insight into Problem

Solving Using Technology

Charlie Cox

Preliminary Oral Defense

Outline

What is IMMEX? Why use IMMEX? Proposed project

What is IMMEX?

Originally developed for the UCLA medical school.

Internet – Based Software Novel Assessment Tool

IMMEX Problems

Case – Based Non-precribed strategies Immediate Assessment

http://ublib.buffalo.edu/libraries/projects/cases/case.htmlWilkerson, L. & Feletti, G. (1989). Problem – Based Learning: One Approach to Increasing Student

Participation, In A.F. Lucas (Ed.), The Department Chairperson’s Role in En hancing College

Teaching, New Directions for Teaching and Learning, 37, 51 – 66.

Prolog

Problem Space

What is unique about IMMEX?

Immediate feedback Tracks HTML movements

Search Path Map

Search Path Map

Strategy Interpretations

Most commonly 3 strategy - types can be identified: Limited Efficient Prolific

Artificial Neural Networks

Hidden Markov Models

Limited Search Path Map

Additional Example

Prolific Search Path Map

Efficient Search Path Map

Strategy Analysis using ANNs

Pattern Recognition Uses a 6 x 6 grid for pattern recognition

Stevens, R.H., Ikeda, J., Casillas, A., Palacio-Cayetano, J., and Clyman, S., (1999) Artificial Neural

Network-Based Performance Assessments, Computers in Human Behavior, 15: 295 – 314.

Stevens, R. and Palacio-Cayetano, J., (2003) Design and Performance Frameworks for Constructing

Problem-Solving Simulations, Cell Biology Education, 2: 162 – 179.

A Node From Hazmat

0

0.2

0.4

0.6

0.8

1

1.2

Problem Space Item

P

R

O

B

A

B

I

L

I

T

Y

Background

Flame, Solubility,Conductivity

Litmus

HCl,NaOH

Precipitins

Hidden Markov Modeling

Used to study student transitions when working problems.

Predictive Modeling

Soller, A., and Lesgold, A. (2003). A Computational Approach to Analyzing Online KnowledgeSharing Interaction. Proceedings of Artificial Intelligence Education, 2003, Australia. 253.

HMM Probabilities

Initial

Transition

aij = Pr[qt+1=Sj|qt=Si]

Emission or Observational Probabilities

bj(k) = Pr(vk at t|qt = Si]

Stevens et. al. ITS, 2004, accepted

Transition Table

To State:

1 2 3 4 5

1 0.99 0.00 0.00 0.00 0.00

2 0.05 0.55 0.02 0.07 0.31

From State: 3 0.06 0.34 0.22 0.33 0.05

4 0.03 0.01 0.01 0.94 0.01

5 0.02 0.03 0.00 0.00 0.95

Stevens et. al., ITS, 2004, accepted

Transition Summary

Stevens et. al., ITS, 2004, accepted

Framing, Transitioning, Stabilization

StateStrategy

Sequence Description Trajectory

1 32 33 28 33 33

Limited test selections, few background resources Localized

2 12 18 24 20 1

Many tests becoming fewer with progress. Progressive

3 5 24 6 18 Many test selections. Localized

4 4 22 33 33Resource extensive

going to data. Shifting

5 4 6 14 25 30 19

Shifts between data rich and data lean strategies. Shifting

Item Response Theory

Examines each item individually as it is added and removed.

1 - parameter logistic Student ability Item Difficulty

€

logP(xj=1P(xj=0 ⎛ ⎝ ⎜ ⎜

⎞ ⎠ ⎟ ⎟=θ−bj

Research Objectives

Does teaching style influence problem solving?Passive vs. Active Methods

IRT and States will be compared

Cooper, M.M. (1995), Cooperative Learning: An Approach for Large Enrollment Courses Journal of Chemical Education. 162.

Research Objectives

• Stabilization• Gender Effects• Period of Time for Stabilization

Research Objectives

The Effect of Formal Reasoning on Problem Solving using the GALT test.

Spatial Ability

Bunce, Diane M.; Hutchinson, Kira D. J. Chem. Educ. 1993, 70, 183.Herron, D. The Chemistry Classrom: Formulas for Successful

Teaching, American Chemical Society, Washington DC, 1995.

Transfer of Knowledge

Problem Similarities Teaching Style Gender

IMMEX Problems for the Study

In - Stereo OrganoMech Lewis Structure Finding Carbon’s Neighbors TLC Separation Specta Analysis

In - Stereo Figures

Line Structure

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Lewis Structure



Newman Projection



Sawhorse Projection

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

3 - D Ball and Stick Figure



3 - D Wireframe



3 - D Stick Model



Plugin

Rotation in 3 - D space Switch between the possible 3 - D figures

while rotating.

TLC Problem -- Unknown Analysis

Unknown identification based upon TLC analyses.







Additional Information

Developing Solvent Stationary Phase Solvent Volume Visualization Method Library

Rf Values Examples of TLC of a Known Compound

Spectra Analysis

1H, 13C, IR, and MS data is available for students to elucidate an unknown’s structure.



Spectra Analysis Continued



Additional Spectra Analysis INFO

1. Example Spectra a. Benzaldehyde b. 2-butanone c. Benzene d. Butylamine e. Propionic Acid2. Correlation Tables3. Library

Preliminary Results

Lewis Structure (Male and Female Strategies)State

1.00 2.00 3.00 4.00 5.00 Total

Count 66 11 6 38 9 130ExpectedCount

58.2 7.8 20.2 28.3 15.6 130.0

Female

% withinGender

50.8% 8.5% 4.6% 29.2% 6.9% 100.0%


120.8 16.2 41.9 58.7 32.4 270.0

Gender

Male

% withinGender

41.9% 4.8% 20.7% 18.1% 14.4% 100.0%


179.0 24.0 62.0 87.0 48.0 400.0

Total

% withinGender

44.8% 6.0% 15.5% 21.8% 12.0% 100.0%

p < 0.000

Lewis Structure Performance

Correct

.00 1.00 Total

Count 49 81 130ExpectedCount

54.3 75.7 130.0

Female

% withinGender

37.7% 62.3% 100.0%


112.7 157.3 270.0

Gender

Male

% withinGender

43.7% 56.3% 100.0%


167.0 233.0 400.0

Total

% withinGender

41.8% 58.3% 100.0%

p = 0.151

Stabilization

0.00 1.00

Bars show Means

1.00 2.00 3.00 4.00 5.00

State

1.00

2.00

3.00

4.00

Step

1.00 2.00 3.00 4.00 5.00

State

Female Male

Spectra Analysis (GPA & State)

1 2 3 4 5

State

2.00

2.50

3.00

3.50

4.00

GPA

??

??

? ?

State Description1 Random Type Testing. Around 50% of the students used prolific type strategies

and view excessive information while others viewed little information2 Involved normal testing with extensive use of available hints and library materials.3 Normal testing with the use of available hints and little use of examples and the

library.4 Involved increased use of examples and hints.5 This is described as more random testing with some use of all materials, but not

in a very high frequency (i.e. ~50%)

Sum ofSquares df Mean Square F Sig.

Between Groups 4.950 4 1.237 6.421 .000Within Groups 63.212 328 .193Total 68.162 332

Interventions -- Collaborative Grouping

Anova: SingleFactor

SUMMARYGroups Count Sum Average Variance

Before 9504.0000 385836.0900 40.5972 93.7453Group 3901.0000 161179.7300 41.3175 127.1533After Group 8727.0000 358399.1600 41.0679 102.2055

ANOVASource ofVariation SS df MS F P-value F crit

Between Groups 1795.0939 2.0000 897.5470 8.7167 0.0002 2.9961Within Groups 2278604.4645 22129.0000 102.9692

Total 2280399.5584 22131.0000

State Distribution -- Collaborative Grouping

State 1 2 3 4 5 Totals

Observed 390 236 167 41 14Prior to grouping

Expected 37308 282.7 138.6 33.7 19.3848

Observed 287 276 84 20 21Post grouping

Expected 303.2 229.3 112.4 27.3 15.7688

Totals 677 512 251 61 35 1536(Chi Square=38.624, p<0.001)

In - Stereo

Documents

IMMEX: Providing Insight into Problem Solving Using Technology Charlie Cox Preliminary Oral Defense