TeachingWithData.org Resources for Teaching

Quantitative Literacy in the Social Sciences

John Paul DeWitt & Lynette HoelterUniversity of Michigan

ASA Annual Meeting, August 15, 2010

Presentation Outline:

• Introducing the project partners• Quantitative Literacy • Introducing TeachingWithData.org

– General overview (demo of Website)– Sociology-related resources– Future directions

Project Partners• ICPSR • SSDAN• Others involved:

– American Economic Association Committee on Economic Education

– American Political Science Association– American Sociological Association– Association of American Geographers– Science Education Resource Center, Carleton

College

ICPSR

• World’s oldest and largest social science data archive– Began in 1962 as ICPR

• Membership organization with 700+ members worldwide (non-members can use many resources)

• Summer Program in Quantitative Methods of Social Research

Current Snapshot of ICPSR• Currently 7,880 studies (65,200 data sets)

– Grouped into Thematic Collections– Available in multiple formats– Federal funding allows parts of the

collection to be openly available– Data sources:

• Government• Large data collection efforts• Principal Investigators• Repurposing• Other organizations

ICPSR: Undergraduate Education

• Fairly recent attention– Response to faculty– Undergrad users are fastest growing

segment

• Resources– OLC, SETUPS, ICSC, EDRL

• NSF-funded projects– TeachingWithData.org (NSDL)– Course, Curriculum, & Laboratory

Improvement project to assess the effect of using digital materials on students’ quantitative literacy skills

7

SSDAN-OLC

• SSDAN’s primary focus is to assist in the dissemination of social data into the classroom with sites like DataCounts! and CensusScope

• ICPSRgreat track record in research, with a new attention on undergraduate education coming more recently with the welcomed Online Learning Center (OLC)

8

SSDAN: Background• Started in 1995• University-based organization that creates

demographic media and makes U.S. census data accessible to policymakers, educators, the media, and informed citizens. – web sites– user guides – hands-on classroom materials

• Integrating Data Analysis (IDA)

9

SSDAN: Classroom Products

• DataCounts! (www.ssdan.net/datacounts)– Collection of approximately 85 Data Driven Learning

Modules (DDLMs)– WebCHIP (simple contingency table software)– Datasets (repackaged decennial census and

American Community Survey)– Target audience is lower undergraduate courses

• CensusScope (www.censusscope.org)– Maps, charts, and tables – Demographic data at local, region, and national levels– Key indicators and trends back to 1960 for some

variables

10

SSDAN: DataCounts!

Quickly connects users to datasets…

..or Data Driven Learning Modules

11

SSDAN: DataCounts!

Menu for choosing a dataset for analysis

Brief List of available dataset collections

12

SSDAN: DataCounts!Submitting a module:• Sections are clearly laid out• Forces faculty to create modules

with specific learning goals in mind.

• Makes re-use of module much easier

13

SSDAN: DataCounts!

TitleAuthor and Institution

Brief Description

Faceted browsing to refine the search• Appropriate Grade Levels• Subjects (e.g. Family, Sexuality and

Gender)• Learning Time

14

SSDAN: DataCounts!Data Driven Learning Modules are clearly laid out• Easy to read• Instructors can quickly identify

whether a module would be relevant to a specific course

15

SSDAN: DataCounts!

• WebCHIPCommands for selecting variables, creating tables, graphing, and recoding

Basic information about the dataset

Running the “marginals” command shows the categories for each variable and frequencies

16

SSDAN: DataCounts!

Students can quickly run simple cross tabulations to see distributions and test hypotheses

17

SSDAN: DataCounts!

Controlling for an additional variable allows for deeper analysis

18

SSDAN

• DataCounts!– Collection of approximately 85 Data Driven Learning

Modules (DDLMs)– WebCHIP (simple contingency table software)– Datasets (repackaged decennial census and

American Community Survey)– Target is lower undergraduate courses

• CensusScope– Maps, charts, and tables – Demographic data at local, region, and national levels– Key indicators and trends back to 1960 for some

variables

19

SSDAN: CensusScope

New ACS data with improved look & feel coming Fall 2010

20

SSDAN: CensusScope• Charts, Trends,

and Tables• All available for

states, counties, and metropolitan areas

Thinking about Quantitative Literacy (QL)

• CCLI project to measure effectiveness of using online modules to teach QL

• First need to agree on skill set representing QL in the social sciences– Most use data-based exercises to teach

content– QL/QR has gotten much recent attention

in institutional assessment, many schools requiring a QL component

What is QL?• “Statistical literacy, quantitative literacy, numeracy --

Under the hood, it is what do we want people to be able to do: Read tables and graphs and understand English statements that have numbers in them. That’s a good start,” said Milo Schield, a professor of statistics at Augsburg College and a vice president of the National Numeracy Network.

Shield was dismayed to find that, in a survey of his new students, 44 percent could not read a simple 100 percent row table and about a quarter could not accurately interpret a scatter plot of adult heights and weights.

Chandler, Michael Alison. What is Quantitative Literacy?, Washington Post, Feb. 5, 2009

Similar to Critical Thinking:

• Students as participants in a democratic society

• Skills include:– Questioning the source of evidence in a

stated point– Identifying gaps in information– Evaluating whether an argument is

based on data or opinion/inference/pure speculation

– Using data to draw logical conclusions

Quantitative Literacy

• Necessary for informed citizenry• Skills learned & used within a context• Skills:

– Reading and interpreting tables or graphs and to calculating percentages and the like

– Working within a scientific model (variables, hypotheses, etc.)

– Understanding and critically evaluating numbers presented in everyday lives

– Evaluating arguments based on data– Knowing what kinds of data might be useful in answering

particular questions

• For a straightforward definition/skill list, see Samford University’s (not social science specific)

Translating to Learning Outcomes

• Began with AAC&U rubric for quantitative reasoning• QL in social sciences:

– Calculation– Interpretation– Representation– Analysis– Method selection– Estimation/Reasonableness checks– Communication– Find/Identify/Generate data– Research design– Confidence

Learning Outcome Dimensions

• Calculation: Ability to perform mathematical operations

• Interpretation: Ability to explain information presented in a mathematical form (e.g., tables, equations, graphs, or diagrams)

• Representation: Ability to convert relevant information from one mathematical form to another (e.g., tables, equations, graphs or diagrams)

• Analysis: Ability to make judgments based on quantitative analysis

Learning Outcomes (con’t)

• Method selection: Ability to choose the mathematical operations required to answer a research question

• Estimation/Reasonableness Checks: Ability to recognize the limits of a method and to form reasonable predictions of unknown quantities

• Communication: Ability to use appropriate levels and types of quantitative information (data, reasoning, tools) to support a conclusion or explain a situation in a way that takes the audience into account.

Learning Outcomes (con’t)

• Find/Identify/Generate Data: Ability to identify or generate appropriate information to answer a question

• Research design: Understand the links between theory and data

• Confidence: Level of comfort in performing and interpreting a method of quantitative analysis

29

Assessment Tools and Results

QL Skills Are Marketable

• Often cited by students as something “tangible” that they have learned

• Definable skill set useful in many career paths

• Easy to tie to everyday life

Including Data Builds QL and:

• Engages students with disciplines more fully – Active learning– Better picture of how social scientists

work– Prevents some of the feelings of

“disconnect” between substantive and technical courses

• Piques student interest• Opens the door to the world of data

TeachingWithData.org

• National Science Digital Library – only social science pathway

• Goal: Make it easier for faculty to use real data in classes– Undergraduate (esp. “non-methods”)– K(9)-12 efforts

• Includes survey of ~3600 social science faculty • Repository of data-related materials

– Exercises, including games and simulations– Static and dynamic maps, charts, tables– Data – Publications

• Tagged with metadata for easy searching

Major Changes since Oct. 2009

• Redesign of the interface on the main page– Guided Search from home page– Resources categorized by more general ‘resource type’ controlled vocabulary

• Data focused on tables and figures vs. data sets• Reference Shelf Data Sources, events, pedagogy• Classroom Resources Grouped like resources,

– Search box with grade level

• Spring Cleaning – removed hundreds of resources• Identified items at lower levels (higher granularity)• User log-in (OpenID) and submission• Local content• Data in the News blog• Data for Online Analysis• Reading list: ability to create, save, and share

– Favorites– List of resources for course, project, or textbook– TwD and external resources

New Account Setup (OpenID)

New Account Setup

TeachingWithData.org

TeachingWithData.org

TeachingWithData.org

TeachingWithData.org

Future Changes

• Professional Association editors– Submit, edit metadata, review resources

• “Report” button for review and edit– Cleaner metadata, outdated links, etc

• Comments• OpenStudy partnership?

– Ratings– Recommendations– User Collaborations (Instructor-Instructor, Instructor-

Student)– Instant feedback and help

– TRAILS indexing

OpenStudy.com

Sociology Resources

Example Resources

• “Data in the News” feature – good way to bring in current events

• Lesson plans/lectures• Data-driven exercises• Data sources• Tools

Lesson Plans (Example)

More Extensive Lesson Plans (Example)

International Data & Information for Comparison (Example)

Example: Short Video on Family Change in Canada

Static Tables (Example)

Graphs & Maps (Example)

Interactive Maps (Example)

Data-Based Exercises: “Low-Tech” (Example)

Data-Based Exercises: Online (Example)

Data-Based Exercises: No Stat Software Needed (Example)

Simulations (Example)

Data for Online Analysis: No Software Needed (Example)

Educational Data Extracts for Statistics Packages (Example)

Tools for Data Visualization (Example)

Future Directions:

• Include resources for high school teachers

• Ability to link data to analysis and/or visualization tools

• Ability for faculty to rate and comment on resources

• Peer-reviewed materials and capability for faculty to upload their own resources

• Community building through professional associations and networks of users

Your Turn!

• What have you tried? • What has worked best? • Favorites we should include in TwD?

Acknowledgements

• PI: George C. Alter, ICPSR• Co-PI: William H. Frey, SSDAN

• Funded by National Science Foundation grant DUE-0840642