Dr. Wendy L. Keeney-KennicuttDr. Wendy L. Keeney-KennicuttPresidential Professor for Teaching Excellence, Piper Professor of Presidential Professor for Teaching Excellence, Piper Professor of

20102010Associate Director, First Year Chemistry Program, TAMUAssociate Director, First Year Chemistry Program, TAMU

k-keeney@tamu.eduk-keeney@tamu.edu

Ms. Zahira MerchantMs. Zahira MerchantEducational Psychology, TAMUEducational Psychology, TAMU

zhmerchant@neo.tamu.eduzhmerchant@neo.tamu.edu11

Using Virtual 3-D Technology toImprove Learners’ Spatial and

Science Skills: Teaching VSEPRTheory in Second Life®®

A STUDYA STUDY

PremisePremise

22

to investigate if Second Life can enhanceto investigate if Second Life can enhance

– learning of a chemistry concept:learning of a chemistry concept:

3-D shapes of molecules (VSEPR* theory)3-D shapes of molecules (VSEPR* theory)– Spatial ability – translating 3-D images into Spatial ability – translating 3-D images into

2-D pictures and back again 2-D pictures and back again

* Valence Shell Electron Pair Repulsion Theory* Valence Shell Electron Pair Repulsion Theory

My Second Life SpaceMy Second Life Spaceon 12on 12thth Man Island Man Island

(maps.secondlife.com/secondlife/12th%20Man/215/237/(maps.secondlife.com/secondlife/12th%20Man/215/237/26)26)

33

Classroom Use

44

Classroom Area – for reviews and office hoursClassroom Area – for reviews and office hours

BlackboardMovie Screen

Streaming DesktopClickerSystem

QuizSystem

Seats that allow students to raise their hands

Notice Board

TAMU IslandsTAMU Islands

55

Dr. K’s Chemistry

Corner

Aggie Orientation

TAMU Faculty are using SL to do Meetings Simulations

HSC Nursing*Ag Crisis Comm*Ecosystems*

Student Talks* Guest speakers Tech Writing Office hours & Reviews Learning modules Pre-Teacher training* Architecture Projects Streaming live events Viz lab student films* Funded projects

TAM Health Science Center (4 Islands)

Agri-Culture

TAMU-Galveston(2 Islands)

Eco-system

CommunicationIsland - coming

Study DesignStudy Design

Quasi-experimental pre-posttest control group Quasi-experimental pre-posttest control group research design studyresearch design study– 2 classes of Chem 101 students with same2 classes of Chem 101 students with same

instructor (me)instructor (me)– ~ 240 students in each class~ 240 students in each class – all assessments with HW worth 40/700 pts.all assessments with HW worth 40/700 pts.

Experimental group: 3 activities in SL (6wks)Experimental group: 3 activities in SL (6wks) Control group: same 3 activities using two 2-Control group: same 3 activities using two 2-

D rotated images (SL screen shots)D rotated images (SL screen shots) See blog: chemist-in-sl.blogspot.comSee blog: chemist-in-sl.blogspot.com

66

Study DesignStudy Design Blog: chemist-in-sl.blogspot.comBlog: chemist-in-sl.blogspot.com

77

Pre-TestingPre-Testing Spatial AbilitySpatial Ability

– Card Rotations Test (2-D) from ETS: paperCard Rotations Test (2-D) from ETS: paper Educational Testing Service http://www.ets.org/Educational Testing Service http://www.ets.org/

– Purdue Spatial Visualization Test (3-D): onlinePurdue Spatial Visualization Test (3-D): onlineBodner, George M, and Roland B Guay. “The Purdue Visualization of Rotations Test.” The Chemical Educator 2.4 (1997) : 1-17.

Chemistry Content Chemistry Content – VSEPR theory: online VSEPR theory: online

Science logicScience logic– Test of Logical Thinking (TOLT): onlineTest of Logical Thinking (TOLT): online

http://ken.tobinweb.net/Papers/TOLT.pdfhttp://ken.tobinweb.net/Papers/TOLT.pdf

88

Pre-TestingPre-Testing

On-line tests prepared by us with On-line tests prepared by us with QualtricsQualtrics

PSVT (20 items, 10 min) TOLT (10 items, 38 PSVT (20 items, 10 min) TOLT (10 items, 38 min)min)

99

Free! Lots of research uses them!

Pre-TestingPre-Testing

CRT CRT – 20 items with 8 parts each20 items with 8 parts each– 10 items per page, 3 min per page10 items per page, 3 min per page– Students must identify if each figure, when compared to the main Students must identify if each figure, when compared to the main

figure on the left, is thefigure on the left, is the Same - S (rotated in the plane), orSame - S (rotated in the plane), or Different – D (flipped over and rotated)Different – D (flipped over and rotated)

1010

Pre-TestingPre-Testing

VSEPR test – developed by meVSEPR test – developed by me– Tested on 3 faculty and ~100 past studentsTested on 3 faculty and ~100 past students– 36 questions, 4 parts, 45 min36 questions, 4 parts, 45 min

(angle, content knowledge, shape id, chemical examples (angle, content knowledge, shape id, chemical examples of shape)of shape)

– Reliability: 0.90 (Cronbach’s alpha) based on post test Reliability: 0.90 (Cronbach’s alpha) based on post test datadata

1111

Post-TestingPost-Testing

Spatial AbilitySpatial Ability– Card Rotations TestCard Rotations Test– Purdue Spatial Visualization TestPurdue Spatial Visualization Test

Chemistry ContentChemistry Content– VSEPR theory on-line testVSEPR theory on-line test

Presence questionnaire + DemographicsPresence questionnaire + Demographics– Included qualitative aspectIncluded qualitative aspect

HW & Laboratory module on VSEPR theoryHW & Laboratory module on VSEPR theory Exam questions - same for both groupsExam questions - same for both groups

1212

Intervention #1Intervention #1The Molecule GameThe Molecule Game ExperimentalExperimental

– Develop SL skills beyond SLDevelop SL skills beyond SL&TAMU orientations &TAMU orientations (inventory, chat, (inventory, chat, interacting with interacting with objects, taking objects, taking photos)photos)

– Begin seeingBegin seeingmolecules in 3-Dmolecules in 3-D

ControlControl– Answer same questions from SL screen shots (2 views)Answer same questions from SL screen shots (2 views)

1313

Intervention #1Intervention #1The Molecule GameThe Molecule Game ControlControl

– Students answer the same questions using SL screen shots (2 Students answer the same questions using SL screen shots (2 views)views)

1414

Intervention #1 Intervention #1 FeedbackFeedback

The Molecule GameThe Molecule Game

1515

87%87% written directions written directions were clear were clear (N=71)(N=71)

69%69% watched video & watched video & found it found it helpful (N=67)helpful (N=67)

86%86% MG is useful to learnMG is useful to learnSL skills (N=49)SL skills (N=49)

85%85% MG ran smoothly MG ran smoothly (N=13)(N=13)

The Chemist as The Chemist as ArtistArtist

ExperimentalExperimental– Develop more SL skillsDevelop more SL skills– Interact and draw 2 viewsInteract and draw 2 views

each of 3each of 3 molecules in molecules in

sandbox with photossandbox with photos

(link, move, copy, rotate)(link, move, copy, rotate)

⇒⇒ ControlControl

– Draw given 2 views of 3 molecules Draw given 2 views of 3 molecules

(5 different sets – screen shots)(5 different sets – screen shots)1616

Intervention #2Intervention #2

The Chemist as ArtistThe Chemist as ArtistControl - Draw given 2 views of 3 molecules (5 different sets – screen shots)

1717

Intervention #2Intervention #2

Intervention #3Intervention #3Tower of VSEPRTower of VSEPR ExperimentalExperimental

– Rez 11 moleculesRez 11 molecules– Measure bond angles, Measure bond angles,

determine geometries, determine geometries,

Lewis dot structures,Lewis dot structures,

draw structuresdraw structures

1818

ControlControl– Do same homework from 2 Do same homework from 2

views of each SL molecule (5 views of each SL molecule (5 sets)sets)

Intervention #3Intervention #3Tower of VSEPR Tower of VSEPR

HWHW

1919

Student AcceptanceStudent Acceptance

Student Issues Student Issues Solutions Solutions– On-line test compliance On-line test compliance gave gave

pts/reminderspts/reminders

– – SL learning curve SL learning curve videos/ppt in SL/RL videos/ppt in SL/RL

– not virtual world savvy not virtual world savvy SL/RL office SL/RL office hourshours

– some think SL is creepy some think SL is creepy TAMU Island TAMU Island

2020

Before interventions, the control (C) and Before interventions, the control (C) and experimental (E) groups were the same as experimental (E) groups were the same as measured by:measured by:

Preliminary ResultsPreliminary Results

2121

TESTC (N=137)

Mean (SD)

E (N=153)

Mean (SD)

TOLT (max=10) 6.43 (2.10) 6.14 (2.24)

PVRT (max=20) 12.36 (3.48) 11.86 (3.70)

VSEPR Content Test (max=45)

7.08 (3.56) 7.39 (3.77)

The two groups were different on CRT pre-test The two groups were different on CRT pre-test but NOT post-test. The E group showed a smallbut NOT post-test. The E group showed a small

but significantly larger gain.but significantly larger gain.

Preliminary ResultsPreliminary Results

2222

CRT(max=160

)

Control Mean (SD)

ExperimentalMean (SD)

p

Pretest 107.9 (30.4) 100.3 (27.9) 0.027 (2-tailed)

Posttest 127.2 (29.5) 123.8 (22.5) 0.28

Gain(over 6 wks)

19.3 (20.4)p<0.001Cohen’s d=0.64

23.5 (17.8)p<0.001Cohen’s d=0.93

0.029 (1-tailed)Cohen’s d=0.22

PVRT PVRT – no significant differences were seen between the – no significant differences were seen between the pretest and posttest for either group.pretest and posttest for either group.

Exam 3 score on VSEPR Exam 3 score on VSEPR – no significant differences – no significant differences

VSEPR HW and Laboratory VSEPR HW and Laboratory – no significant differences– no significant differences

VSEPR content testVSEPR content test– Overall, no significant differences were seen on posttestOverall, no significant differences were seen on posttest

Control: 18.41 ± 7.60 Experimental: 18.67 ± 8.73 Control: 18.41 ± 7.60 Experimental: 18.67 ± 8.73

Preliminary ResultsPreliminary Results

2323

More on VSEPR test: More on VSEPR test: On Part 1 On Part 1 (determining angle from drawing), the (determining angle from drawing), the E group did significantly better (max=4)E group did significantly better (max=4)

Preliminary ResultsPreliminary Results

2424

VSEPR Pt 1(max=4)

Control Mean (SD)

ExperimentalMean (SD)

p

Pretest 1.01 (0.57) 1.07 (0.47) 0.26 (2-tailed)

Posttest 1.74 (1.20) 2.01 (1.19) 0.03 (1-tailed)Cohen’s d=0.24

Gain(over 6 wks)

0.74 (1.20)p=0.004Cohen’s d=0.78

0.94 (1.14)p<0.001Cohen’s d=1.04

0.075 (1-tailed)Cohen’s d=0.18

Even more on VSEPR test: Even more on VSEPR test: If you includeIf you includeall pre/post students, the results are evenall pre/post students, the results are evenmore significant, BUTmore significant, BUT

On average, students are recognizing <2/4 of anglesOn average, students are recognizing <2/4 of angles..

Preliminary ResultsPreliminary Results

2525

VSEPR Pt 1(max=4)

Control Mean (SD)

ExperimentalMean (SD)

p

Pretest 1.03 (0.58) 1.04 (0.51) 0.82 (2-tailed)

Posttest 1.55 (1.17) 1.82 (1.19) 0.019 (2-tailed)Cohen’s d=0.24

Gain(over 6 wks)

0.52 (1.17)P<0.001Cohen’s d=0.56

0.78 (1.16)p<0.001Cohen’s d=0.85

0.023 (2-tailed)Cohen’s d=0.22

Let’s look at the questions one by one.Let’s look at the questions one by one.

Here is theHere is the

question verbage:question verbage:

Preliminary ResultsPreliminary Results

2626

Gain-C (-0.04 ± 0.49; N=194)Gain-C (-0.04 ± 0.49; N=194) Gain-E (0.005 ± 0.44; N=209)Gain-E (0.005 ± 0.44; N=209)

Question 1 of 4 Question 1 of 4

2727

p < 0.16(1-tailed)

Confidence–Pre C=74% E=75%

Confidence–Post C=77% E=82%

Gain-C (0.25 ± 0.47)Gain-C (0.25 ± 0.47) Gain-E (0.33 ± 0.49)Gain-E (0.33 ± 0.49)

Question 2 of 4 Question 2 of 4

2828

p < 0.043 (1-tailed)

Confidence–Pre C=53% E=72%

Confidence–Post C=75% E=76%

Confidence–Pre C=58% E=75%

Confidence–Post C=63% E=60%

Gain-C (0.08± 0.44)Gain-C (0.08± 0.44) Gain-E (0.15 ± 0.49)Gain-E (0.15 ± 0.49)

Question 3 of 4 Question 3 of 4

2929

p < 0.064 (1-tailed)

Confidence–Pre C=61% E=59%

Confidence–Post C=65% E=62%

Confidence–Pre C=46% E=57%

Confidence–Post C=74% E=75%

Gain-C (0.23± 0.49)Gain-C (0.23± 0.49) Gain-E (0.29 ± 0.47)Gain-E (0.29 ± 0.47)

Question 4 of 4 Question 4 of 4

3030

p < 0.086 (1-tailed)

Confidence–Pre C=34% E=42%

Confidence–Post C=75% E=74%

Confidence–Pre C=68% E=66%

Confidence–Post C=70% E=68%

Disheartening!!Disheartening!!

CommentsComments

3131

We analyzed two questions posed to We analyzed two questions posed to the Experimental group:the Experimental group:

1. Is it a good idea to use Second Life in 1. Is it a good idea to use Second Life in future future chemistry classes?chemistry classes?

(Quantitative analysis)(Quantitative analysis)

2. Please explain your answer.2. Please explain your answer.

(Qualitative analysis)(Qualitative analysis)

More Preliminary ResultsMore Preliminary Results

3232

3333

Is it a good idea to use Second Life Is it a good idea to use Second Life inin

future classes? future classes?

0

5

10

15

20

25

30

35

Strongly Agree Agree Neutral Disagree Strongly Agree

11%

30%

23%

18%

18%

N = 219

Per

cent

age

Please Explain Your Answer….Please Explain Your Answer…. Main ThemeMain Theme - Perceived Usefulness- Perceived Usefulness

0

10

20

30

40

50

60

70

Yes No Unsure

Perceived Usefulness

N = 100

58%

35%

7%

Subthemes:

1. Visualization (38%)

2. Time Consumption (29%)

3. Interactivity (17%)

4. Other methods such as text books, 2-D images, paper pencilwere just as useful (16%)

3434Other themes: Perceived ease of use & gaming experience

Results - Perceived UsefulnessResults - Perceived Usefulness

Comments1. It helped a lot with the VSEPR theory by allowing me to visualize the molecules so when I’m drawing them on paper I can see the molecule in my mind’s eye. This allowed me to draw 3D molecule faster and easier.

2. I found that it took a lot of time to learn how to manipulate the molecules in Second Life.

3. It allows you actually see and interact with a molecule

4. I feel it would be easier to lecture about this material and show students more examples. 3535

Future Work/QuestionsFuture Work/Questions

Did my attempt to showDid my attempt to show the correct molecular shape the correct molecular shape by giving 2 views of a by giving 2 views of a molecule to the control group molecule to the control group skew the CRT results? skew the CRT results?

Do students taking a regular general chem class Do students taking a regular general chem class increase their CRT scores naturally?increase their CRT scores naturally?

If so, do they keep their higher CRT scores over If so, do they keep their higher CRT scores over time?time?

Can we really assume that studentsCan we really assume that students can see that this angle is 90can see that this angle is 90oo, , without training them? Not!without training them? Not! 3636

AcknowledgementsAcknowledgements

Zahira and I thank all my students Zahira and I thank all my students who had to endure all our who had to endure all our assessments….assessments….

Thank you and any questions?Thank you and any questions?

3737