Authentic Discovery Projectsin Statistics

GCTM ConferenceOctober 16, 2009

Dianna Spence

NGCSU Math/CS Dept, Dahlonega, GA

NSF Grant Project Overview• Grant Title:

“Authentic, Career-Specific Discovery Learning Projects in Introductory Statistics”

• Project Goals: Increase students’... knowledge & comprehension of statistics perceived usefulness of statistics self-beliefs about ability to use and understand

statistics

• Tasks: Develop Instruments Develop Research Constructs and Projects Develop Materials and Train Instructors Measure Effectiveness

Instructional Model: Discovery Learning in Statistics• Authentic Research Projects

Experiencing the Scientific Method

Discovering Statistical Methods in Context• Design of Research Question

• Definition of Variables

• Demographic Data

• Representative Sampling Issues

• Data Collection

• Appropriate Analyses of Data

• Interpretation of Analyses

Project Format

• Linear regression Variables

• student selects• often survey

based constructs Survey design Sampling Regression analysis

• t-tests Variables

• may use data previously collected

Designs• Independent

samples• Dependent

samples Hypotheses

Interdisciplinary Team

• Disciplines Represented Biology/Ecology Criminal Justice Psychology Sociology

• Tasks of Team Members Identify authentic research constructs Define instrument/measurement of construct Suggest simple statistical research projects

Nursing Physical Therapy Education Business

Online Resource:Instructional Materials Homepage

• Instructor GuideProject overview

• Timelines• Implementation tips• Best practices

Handouts for different project phases

Evaluation rubricsLinks to student resources

http://radar.ngcsu.edu/~rsinn/nsf/

Online Resource:Instructional Materials Homepage

• Student GuideOverall Project Guide

• Help for each project phaseTechnology GuideVariables and Constructs

http://radar.ngcsu.edu/~rsinn/nsf/

Common Questions

• How long will the projects take?

• How do I fit this in with the content I am supposed to teach?

• How should I go about organizing and facilitating these projects?

• What are the student “deliverables”?

• How do I assess the students’ work?

• How much should this count in determining student grades?

Project Phases• Form Teams

• Generate Research Ideas

• Develop Constructs and Variables

• Develop Surveys or Other Instruments

• Project Proposals

• Data Collection

• Data Analysis

• Project Report

• Team Presentations

Finding Time for the Project

• Make the projects the vehicle through which students learn course content

• Selecting projects to use as class examples current students’ projects (work in progress) example project(s) you have made up former students’ projects (when you have them)

• Leverage sample projects Gives students an idea what

should be included in their project Helps students connect course

content and put it in context

Aligning Course Content with Project Phases

Phase Class Topic

Define variables Independent/dependent variablesDevelop surveys Types of biasData collection Sampling methodsData analysis Appropriate statistical analyses

Example: Exploring slope of regression line“One team examined the relationship between

number of vegetable servings consumed in a week and number of hours spent exercising in a week. Their regression equation was ___?___. What is the slope and what does it mean?”

Considerations and Options: Forming Teams• Team Size

• Assigned Members vs. “Pick Your Own”

• Grouping by Common Interests

• Giving Team Members Specific Roles

Considerations and Options: Project Proposals

• Formal vs. Informal

• IRB or Similar Entity

• Other Permissions Required?

• Require instructor approval before data collection begins!

Considerations and Options: Data Collection• Dialog About Sampling

StrategiesRandomStratified Convenience

• Dialog About Representative Samples

• Assist Students with Organization and Data Entry

Considerations and Options: Data Analysis• Linear Regression Projects

ScatterplotCorrelation Coefficient rRegression LineRegression EquationSlopeR2 and Explanatory Value of Model

Height and Shoe Size

y = 0.4198x - 18.921

R2 = 0.7684

4

6

8

10

12

14

55 60 65 70 75 80

Height

Sh

oe

siz

e

Considerations and Options: Data Analysis

• t-Test Comparison ProjectsAppropriate Design

• Two Independent Samples• Dependent Samples/Matched Pairs

Appropriate Hypotheses• One-tailed (and which tail)• Two-tailed

Significance LevelInterpretationImplications of Type I & II Errors

Considerations and Options: Project Report

• Content RequirementsOutlinesTemplatesSample ReportsScoring Rubrics

(Students can use as checklist)• Other Requirements

Writing StandardsSubmission of Technology FilesReflections

Considerations and Options: Team Presentations

• Presentation GuidelinesContent and ScopeAestheticsPace and OrganizationTime Limit & Enforcement

• Audience AccountabilityEvaluations1-2 Sentence Recap of Points

Assessment

• RubricsAdvantages

• Consistency• Manageability• Communicate expectations

Encompass All Project Components• Grade milestones along the way

Explicit vs. HolisticResources for Rubrics

• Use one of ours• Customize your own

Assessment

• Team Member Grades Accountability of Individual Members

• Shared Team Grade• Individual Contribution• Other “Tricks”

• Weight of Projects

Exploratory StudyFall 2007

• Instrument Validation and Concept “Trial Run”

• Based on 10 sections of Introductory Stats

• 4 experimental sections Used authentic discovery projects n=113 participants out of 128 students

• 88% participation rate

• 6 control sections Did not use authentic discovery projects n = 164 participants out of 192 students

• 85% participation rate

Exploratory Results: Content Knowledge• Instrument

21 multiple choice items KR-20 analysis: score = 0.63

• Results control mean: 8.87; experimental mean = 10.82 experimental mean 9 percentage points higher experimental group significantly higher (p < .0001) effect size = 0.59

• Instrument shortened to 18 items for full study

Exploratory Results: Perceived Usefulness of Statistics

• Instrument 12-item Likert style survey; 6-point scale 5 items reverse scored score is average (1 – 6) of all items Cronbach alpha = 0.93

• Results control mean: 4.24; experimental mean = 4.51 experimental group significantly higher (p < .01) effect size = 0.295

• Instrument unchanged for full study

Exploratory Results: Statistics Self-Beliefs• Beliefs in ability to use and understand statistics

• Instrument

15-item Likert style survey; 6-point scale

score is average (1 – 6) of all items

Cronbach alpha = 0.95

• Results

control mean: 4.70; experimental mean = 4.82

difference not significant (1-tailed p = .1045)

effect size = 0.15

• Instrument unchanged for full study

Full Study: Pilot of Developed Materials

• 3 institutions 1 university (6 undergraduate sections)

1 2-year college (2 sections)

1 high school (3 sections)

• 5 instructors

• Quasi-Experimental Design Spring 2008: Begin instructor “control” groups

Fall 08 - Fall 09: “Experimental” groups

Pilot Results• Varied by Instructor

• Overall results given here

• Instrument Perceived Usefulness

• Pretest: 50.42• Posttest: 51.40• Significance: p = 0.208

Self-Beliefs for Statistics• Pretest: 59.64• Posttest: 62.57• Significance: p = 0.032**

Content Knowledge• Pretest: 6.78• Posttest: 7.21• Significance: p = 0.088*

Attitudes and Beliefs• Statistics Self-Beliefs

Self-beliefs improved significantly overall• Significant Gains

– for regression techniques ( p = 0.035 )– for general statistical tasks ( p = 0.018 )

• Little or No Improvement– for t-test techniques ( p = 0.308 )

• Perceived Utility for Statistics Student perceptions of the usefulness of

statistics improved slightly but not significantly No sub-scales on this instrument

• Overall Perceived Utility ( p = 0.208 )

Performance Gains

• Concept Knowledge: 3 Components Regression Techniques

• Moderately Significant ( p = 0.086 ) T-test Usage

• Moderately Significant ( p = 0.097 ) T-test Inference

• No gain

For more information

• Project Website http://radar.ngcsu.edu/~djspence/nsf/

• Instructional Materials Home http://radar.ngcsu.edu/~rsinn/nsf/