Using Data to Improve Learning and Improve Teaching Steven A Ackerman

Using Data to Improve Using Data to Improve Learning and Improve Learning and Improve

TeachingTeaching

Steven A AckermanSteven A Ackerman

OutlineOutline

A bit about me A bit about me A bit about you (well maybe not)A bit about you (well maybe not) The LiteraciesThe Literacies What I do… and why…What I do… and why…

• … … the value of data…the value of data… SummarySummary

A bit about who I amA bit about who I am Faculty in Department of Atmospheric and Oceanic Faculty in Department of Atmospheric and Oceanic

Sciences (Univ. Wisc.-Madison)Sciences (Univ. Wisc.-Madison)

Director of the Cooperative Institute for Meteorological Director of the Cooperative Institute for Meteorological Satellite StudiesSatellite Studies

Research focuses on satellite observations of weather Research focuses on satellite observations of weather and climate.and climate.

Teaching focuses on weather and remote sensing coursesTeaching focuses on weather and remote sensing courses• All levelsAll levels• Also outreachAlso outreach

Outline Again…Outline Again…

Information LiteracyInformation Literacy• K-12K-12• College level intro for non-science College level intro for non-science

majorsmajors Data LiteracyData Literacy

• College level; meteorology majorsCollege level; meteorology majors Statistical LiteracyStatistical Literacy

• College level intro for non-science College level intro for non-science majorsmajors

3-literacies3-literacies

Information literacy requires that an Information literacy requires that an individual (student or teacher) individual (student or teacher) recognize that information is needed recognize that information is needed and be able to locate and evaluate and be able to locate and evaluate the needed information. They also the needed information. They also have to effectively and efficiently have to effectively and efficiently access the information. access the information.


To be data literate, students must be To be data literate, students must be familiar with a variety of tools for familiar with a variety of tools for accessing, manipulating and accessing, manipulating and visualizing data. visualizing data.


Statistical literacy requires that a Statistical literacy requires that a student knows how to use statistics student knows how to use statistics to develop evidence in an argument. to develop evidence in an argument. They need to demonstrate an ability They need to demonstrate an ability to assemble data into a summary to assemble data into a summary form, and know what comparisons to form, and know what comparisons to make between data sets. make between data sets.

A Conceptual Model…A Conceptual Model…

Science LiteracyIn

form

ati

on L

itera

cy

Data

Lit

era

cy

Sta

tist

ical Li

tera

cy


Science Literacy

Info

rmati

on L

itera

cy

Data

Lit

era

cy

Sta

tist

ical Li

tera

cy


Science Literacy

Information Literacy

Data Literacy

Statistical Literacy


Science Literacy

Statistical Literacy Information Literacy

Data Literacy

Information LiteracyInformation Literacy

How do we enable students to know How do we enable students to know they need data, and what type of they need data, and what type of data?data?• One way is through storiesOne way is through stories

Classroom-tested lessons that:

•integrate earth science, social studies, language arts and math; •are adaptable for different grade levels, and •focus on Wisconsin weather, culture and narrative arts

Wisconsin Weather StoriesWisconsin Weather Stories

AC Hi. I'm AC. I'm interviewing my mom, DC. So tell me a little bit about yourself

DC Well, I was born and raised in Columbus, Ohio. I grew up in a family of eight people. Pretty much played music most of my life with my parents. Right now currently in school, going to be a pharmacist some day, working at the library school, plus I work at a Pharmacy and I sing in the band.

Wisconsin Weather StoriesWisconsin Weather Stories

AC So have you ever been in a major weather storm?

DC Actually I have. Back in 1995. I was performing on the Carnival Cruise Line in Florida playing on the M.S. Fantasy and we were about five weeks into the gig when we got caught in Hurricane Aaron.

AC How bad was the weather? DC Pretty bad. The boat was pretty rocky. 144-foot waves. Hit the side of the ship, broke a stabilizer. People were sick. We were confined to our rooms, weren't able to go aboard the ship. They didn't want us wandering the halls. Pretty much for about three days we weren't able to land, to dock, do anything. We circled around the eye of the hurricane 180 miles out and it was not a fun ride.

AC Did you get sick at all?

DC I was very sick. I ended up having to go to the ship doctor. Got a shot. Knocked me out for two days.

AC Where did this take place? Where exactly in the ocean were you?

A satellite view of Hurricane Aaron on August 3, 1995

Where were you on December 31, 1967? You probably wanted to be at Lambeau Field in Green Bay, Wisconsin. This was the site of the National Football League’s (NFL) Championship game between the Green Bay Packers and the Dallas Cowboys. Anyone alive will remember this game known as the historic “Ice Bowl”.

Daily Maximum (Red) and Minimum (Blue) Temperature - December 1967 - Green Bay, WI

Applied to undergrad nonmajorApplied to undergrad nonmajor

Write/Tell a personal weather storyWrite/Tell a personal weather story Grouped students by common storyGrouped students by common story Developed questions Developed questions

• How much…, how many…, when does…How much…, how many…, when does…• Why does…Why does…

Collected dataCollected data Analyzed Analyzed Presented dataPresented data

Approaching a teaching/learning Approaching a teaching/learning problemproblem

How do we learn?

ActivityActivity

ActivityActivity

Learning DiagramLearning Diagram(Conceptual Model)(Conceptual Model)

Input: Input: New stimulus. Can be self generated or New stimulus. Can be self generated or obtained otherwiseobtained otherwise

Reception: Reception: Student’s filter to an input. Student’s filter to an input. Motivation, social and physical environment.Motivation, social and physical environment.

Processing: Processing: Re-working the input by integrating Re-working the input by integrating reception with accumulated experiences and reception with accumulated experiences and knowledgeknowledge

Practice: Practice: Use of re-worked inputUse of re-worked input Feedback: Feedback: Evaluation of re-worked inputEvaluation of re-worked input

Learning DiagramLearning Diagram

Bloom’s Taxonomy

So, how does learning diagram help?So, how does learning diagram help?

Bloom’s Taxonomy

Interactive web pages – contouringInteractive web pages – contouring

http://profhorn.aos.wisc.edu/wxwise/AckermanKnox/

Formative EvaluationFormative Evaluation Students were surveyed regarding the applets using field-Students were surveyed regarding the applets using field-

tested questions from “The Flashlight Project” (Ehrmann tested questions from “The Flashlight Project” (Ehrmann 1997).1997).

82% “believed” or “somewhat believed” that the applets 82% “believed” or “somewhat believed” that the applets helped them to master skills and concepts more quickly helped them to master skills and concepts more quickly than with traditional methods.than with traditional methods.

75% “believed” or “somewhat believed” that the applets 75% “believed” or “somewhat believed” that the applets helped to develop their skills and under-standing better than helped to develop their skills and under-standing better than through traditional reading and pencil-and-paper exercises.through traditional reading and pencil-and-paper exercises.

71% “believed” or “somewhat believed” that the applets 71% “believed” or “somewhat believed” that the applets allowed them to learn more on their own, with less allowed them to learn more on their own, with less assistance from the TA.assistance from the TA.

Assessment of Learning Gains

What we learn on applets?What we learn on applets?

Delayed Recall

• TOTAL understanding F = 3.562, p < .06

• RH as moisture indicator F = 5.062, p < .025

Immediate Recall

• TOTAL understanding F = 4.380, p < .038

• RH-temp relationship F = 49.160, p < .001

• RH as moisture indicator F = 7.898, p < .005

A teaching conceptual model…A teaching conceptual model…


Good TeachingLe

arn

ing T

hro

ug

h

Div

ers

ity

Learn

ing C

om

mun

ity

Teach

ing

as

Rese

arc

h


Good Teaching

Learning through Diversity

Learning Community

Teaching as Research


Good Teaching

Teaching as Research

Learning Community

Learning through Diversity

Why these differences between Why these differences between classes…classes…

The AOS101 meets an extra time during the week and The AOS101 meets an extra time during the week and while students are exposed to new material during this 50 while students are exposed to new material during this 50 minute meeting, these extra meetings may help identify minute meeting, these extra meetings may help identify material in the lecture class that is not fully understood by material in the lecture class that is not fully understood by the student the student

Small group work projects in the discussion section may Small group work projects in the discussion section may benefit the targeted group.benefit the targeted group.

Self selection, there may be a certain population that self Self selection, there may be a certain population that self

selects out of the AOS101 class.selects out of the AOS101 class.

How do we fix this?How do we fix this?

The AOS101 discussion section The AOS101 discussion section focuses more on weather analysis focuses more on weather analysis than in the lecture class. So, going to than in the lecture class. So, going to a hybrid class structure, part lecture, a hybrid class structure, part lecture, part on-line group work, will enable part on-line group work, will enable us to include more real-time weather us to include more real-time weather map discussions and interpretation. map discussions and interpretation.

Addresses: …AOS101 meets an extra time during the weekAddresses: …AOS101 meets an extra time during the week


Including some sort of social Including some sort of social networking tools in the lecture class networking tools in the lecture class may help reduce the adverse may help reduce the adverse outcomes of the targeted minority. outcomes of the targeted minority.

Addresses: … Small group work projects in the discussion Addresses: … Small group work projects in the discussion sectionsection


We are not sure that technologies We are not sure that technologies will impact this explanation of the will impact this explanation of the difference. difference.

Addresses: … Self selectionAddresses: … Self selection

Why use technology?Why use technology?

Why use the Web specifically…Why use the Web specifically… The Web can make teaching more effective.The Web can make teaching more effective. The Web can augment traditional resources.The Web can augment traditional resources. The Web can save time for students and The Web can save time for students and

teachers.teachers.

Why use the Web specifically…Why use the Web specifically… The Web can make teaching The Web can make teaching lessless effective. effective. The Web can The Web can distractdistract traditional resources. traditional resources. The Web can The Web can costcost time for students and time for students and

teachers.teachers.

Why use the Web specifically…Why use the Web specifically… Program with funding…Program with funding…

Why use the Web specifically…Why use the Web specifically… Program with funding…Program with funding…

The issue:

Can we simulate lab based activities/learning in a large lecture for the Introduction to Weather and Climate course.

The course:

Two major groups of studentsAOS 100 No lab section, three credits: n=319AOS 101 Lab section, four credits: n=125Mostly non-science majorsPlurality are sophomores

The solution?

Will on-line activities work?

Photo: Classroom Management

Forecast ContestForecast Contest






Multi-pronged plan to assess impact of activities on learning:

1. Regurgitation questions.

Use on homework, then ask again on testStudents retain relatively well.

“What layer of atmosphere is closest to the earth's surface?”

99.2% of students got this right on homework.81.5% got this right on test

2. Application-based questions

Make students think about what they know, get them to apply it to new situations.

Students are not as good at these.

Testing Services provides an data on each student's response to every single multiple choice question on the exams.

We can also have data on which students visited the optional content activities on the course website.

So... we can write code to parse these two data files to compare performance on exams to completion of and performance on applets.

Example Activity: Precipitation Type

http://profhorn.meteor.wisc.edu/wxwise/precip/precip-wb.html

Which of the following pairs contain two forms of precipitation that both require an inversion near the ground?

a. snow and sleetb. snow and rainc. snow and freezing raind. freezing rain and sleete. sleet and snow

64.9% of students looked at the applet at some point during the semester.

Of those, 85.3% got the answer right on 1st exam90.6% got the answer right on final.

Of the students who did not look at the applet:

75.9% got the answer right on 1st exam84.8% got the answer right on final.

Sample size was 319.

Difference significant at the 1% level

Build your own web-based learning activities with Reusable Content Objects (RCOs)

+

+

Data

XML

Chosen Tool (RCO)

HTMLLearning Activity

+

Tools for instructors to develop customized lessons

Satellite Observations in Science Education (SOSE), with NASA support, has developed tools to demonstrate the quantitative information in satellite imagery.

Build your own web-based learning activities with Reusable Content Objects (RCOs)

+

+

Data

XML

Chosen Tool (RCO)

HTMLLearning Activity

+

Reusable Evaluation Objects (REOs)

In summaryIn summary

That’s a glimpse of what I do, why I That’s a glimpse of what I do, why I do it and the frameworks I use.do it and the frameworks I use.

Data important for learning and Data important for learning and improving teachingimproving teaching

Data is good, but no data is better Data is good, but no data is better than bad data.than bad data.

Documents

Using Data to Improve Learning and Improve Teaching Steven A Ackerman