The Right Questions and Graphical Displays

THE RIGHT QUESTIONS AND GRAPHICAL DISPLAYS

AGENDA OBJECTIVES

Agenda and Objectives

• Descriptive statistics• Dispersion• Aggregate data• The right questions and graphics• Data inference

Compare basic descriptive statistics and identify their limitations

Describe common mistakes associated with analyzing "on average" data

Explain the purpose of the Data Narrative analyses

Evaluate research questions against criteria for quality

In the last activity, we realized our need for disaggregating the data.

In the Data Narrative, we’ll perform two separate disaggregate analyses,

guided by research questions that we pose of the data.

How Does This Relate To The Data Narrative?

This is in your

Handout

Let’s step back and think about research questions that are being

asked of other data.

Research Questions to be “Consumed”

Did people who switched from Geico to 21st Century Insurance save money on

their car insurance premiums?

This is in your

Handout

their car insurance premiums? Were smokers who used

Chantix more likely to be quit on weeks 9 through 12 than smokers who took the

sugar pill?

their car insurance premiums? Were smokers who used

Chantix more likely to be quit on weeks 9 through 12 than smokers who took the

sugar pill?

What’s the

answer? Let’s find

On The Next Slide, Click “Play” On The Screen

Did people who switched from Geico to 21st Century Insurance save money on their car insurance premiums?

On The Next Slide, Click “Play” On The Screen

Do smokers who use Chantix have a higher probability of being quit on weeks 9 through 12 than users who took the sugar pill?

“There is no such thing as a stupid question…

Actually, yes there is.”

What is Wrong With These Questions?

Were smokers who used Chantix more likely to be

quit on weeks 9 through 12 than smokers who took the

sugar pill?

Click ahead when you’ve completed the appropriate section

of your Handout

SAMPLE BIAS

This question is laden with sample bias. It’s only asking about people who switched. Why did people switch from Geico in the first place? Probably because they could save money.

The question didn't say, "for the average person, when shopping for car insurance, was Geico or Century 21 cheaper?"

SAMPLE BIAS

This question is laden with sample bias. It’s only asking about people who switched. Why did people switch from Geico in the first place? Probably because they could save money.

The question didn't say, "for the average person, when shopping for car insurance, was Geico or Century 21 cheaper?"

This is a stupid question. It’s like

asking,

“Do students who attend after-school tutoring do worse than students who never needed to attend tutoring?”

sugar pill?

LEADING QUESTION

sugar pill?

LEADING QUESTION

This is a leading question. They stacked the deck. Why weeks 9 to 12? Why not weeks 6 through 10? More than likely, they retroactively imposed the question when they noticed

that the results were favorable to their position.

The question didn't say, ”in a comparison using a randomized controlled trial, did people who used Chantix quit smoking and stay stopped more than the sugar pill, including people who didn’t quit at all, people who quit and started up again, people who dropped out of the

study, people who quit, started up again, stopped again, and kept going, etc. etc.?"

sugar pill?

LEADING QUESTION

This is a leading question. They stacked the deck. Why weeks 9 to 12? Why not weeks 6 through 10? More than likely, they retroactively imposed the question when they noticed

that the results were favorable to their position.

The question didn't say, ”in a comparison using a randomized controlled trial, did people who used Chantix quit smoking and stay stopped more than the sugar pill, including people who didn’t quit at all, people who quit and started up again, people who dropped out of the

study, people who quit, started up again, stopped again, and kept going, etc. etc.?"

This is a stupid question. It’s like studying your data, noticing some unexpected trend, then

asking a research question about that unexpected trend, already knowing the answer.

The rubric for the Data Narrative does not ask you to disaggregate your data in a way to make you look good. Your research question should

meet its capacity to surprise.

What Makes a Great Research Question?

https://c2.staticflickr.com/8/7116/7453374794_0bab20ec4c_h.jpghttps://upload.wikimedia.org/wikipedia/commons/1/15/Red_Apple.jpg

http://upload.wikimedia.org/wikipedia/commons/3/3a/College_graduate_students.jpg

Minable: Can this Question be Thoroughly Explored?

Did my ELLs in 2nd hour improve on spiraled MC questions from IA #2 to IA #3?

Did my ELLs in 2nd hour improve on spiraled MC questions from IA #2 to IA #3? Not minable.

We don’t want a yes/no question.

Plus it’s too narrow. Only ELLs, only

spiraled MC questions, only IA

#2 vs. IA #3.

How did my students’ performance on spiraled standards vary from assessment to assessment?

Now this is a minable question!

Notice it starts with “How did” not “Did”

Crisp: Is it Succinct and Easy to Understand?

How did boys perform versus girls, overall, across reading and math, relative to question type MC vs OR, and with respect to the different assessments that I administered for my Pathway, including unit and interim?

Not succinct.

We’re not sure what the question is

asking.

What, if any, were the achievement differences between boys and girls in my class?

Now this is a succinct question!

Notice that the question doesn’t lead…it doesn’t

assume to know the answer already

Meaningful: Is There Rationale for Why I Should Care?

Were the trends in academic performance any different for students who preferred for Ruffles potato chips over Lays in our “Harvest Fair Taste Test”?

Not meaningful.

It’s difficult to imagine a rationale for why we should

care about the answer. We don’t

care. We don’t want to know the answer.

How did my students’ performance vary for those in both my class and also in my advisory, versus those who were only in my class?

Now this is a meaningful question! We can envision a solid

rationale for why we should care about

performance of students who spent more or less time with the teacher

Research Question Matchup

Research Question Matchup!

Q1: Did basketball players score higher than actors on standard 6.1.d?

Q2:What (if any) differences in achievement existed between athletes and entertainers for math standards mastery?

of your Handout

Q1: Did basketball players score higher than actors on standard 6.1.d?

Q2 is minable.

Q1 is not. Q1 is a narrow, yes/no question. NOT minable.

Q2 wins the matchup!

Q1: What, if any, variation of achievement existed between people who use a nickname (like “Bill” Clinton, “Steffi” Graf, or “Tony” Blair) and those who didn’t (like “Michael” Bloomberg)?

Q2:What, if any, variation of achievement existed between people who had initials early in the alphabet (like “Bill Clinton” or “Andrew Cuomo”) and those who didn’t (like “Serena Wiliams”or “Justin Timberlake”)?

of your Handout

Q1: What, if any, variation of achievement existed between people who use a nickname (like “Bill” Clinton, “Steffi” Graf, or “Tony” Blair) and those who didn’t (like “Michael” Bloomberg)?

This is a tough matchup.

Both questions seem meaningless.

HOWEVER, there is some research to suggest that people with names earlier in the alphabet are more successful in life because they are placed first in line more often, over time. So you could write a compelling rationale that makes Q2 meaningful to the reader of your Data Narrative. It better be compelling though.

So Q2 wins the matchup!

Q1:Why did some students meet the Proficient Goal (for math Standards Mastery) while some didn’t?

Q2:For my students who did not meet the Proficient Goal (for math Standards Mastery), were there any commonalities in the content that they struggled to master?

of your Handout

Q1:Why did some students meet the Proficient Goal (for math Standards Mastery) while some didn’t?

Q1 is an interesting question, but we don’t have the data we need to answer broad “why” questions. This is a question better-suited for earning your PhD than for writing your Data Narrative.

For Q2 you have the opportunity to find commonalities like standards or question types that these students struggled with the most.

So Q2 wins the matchup!

Research Questions: Your Turn!Take a moment to jot down your ideas.

of your Handout

We’ll also create graphical displays to visualize the findings in response to

the research questions.

“Figures don't lie…

but liars figure.”

What Could be Improved About this Display?

The bar graphs below display students’ overall math Standards Mastery, comparing boys versus girls for Class #1.

of your Handout

Titles!

Unify!

Axes Labels!

Axes Scaling!

What is Better About this Display?

CS CT RL SG LL S LJ AC MB SW0

Overall Standards Mastery, Boys vs. Girls

BoysGirls

Individuals

What is Better About this Display?

How else could we display this data? Ideas?

BoysGirls

Individuals

What Information did we Lose by Aggregating?

Boys Girls0

Avg = 41.5

Avg = 62.75

Avg Standards Mastery, Boys vs. Girls

n = 6n = 4

of your Handout

What Information did we Lose by Aggregating?

Boys Girls0

Avg = 41.5

Avg = 62.75

Avg Standards Mastery, Boys vs. Girls

n = 6n = 4

We can’t see the spread

(dispersion) of scores within those buckets

How Can We Make This Even Better?

CS CT RL LJ AC MB SG LL S SW0

BoysGirls

Individuals

BoysGirls

Individuals

eA Super Rich Graphic: Four Dimensions!(super rich, get it?)

70% - Achieve Fl.80% - Amb. Goal

BoysGirls

Individuals

eA Super Rich Graphic: Four Dimensions!

70% - Prof. Goal80% - Amb. Goal

Hope you’re impressed.McKayla Maroney.

55% Standards Mastery.

Class #2.

The Right Questions and Graphical Displays

Education

Use of Animation for Enhancing Mathematical Graphical Displays

All LED displays are basically the same...Right?

4 Graphical Presentation of Data - mas.ncl.ac.uknlf8/teaching/mas1343/notes/chap4-5.pdf · 4 Graphical Presentation of Data 4.1Introduction Graphical displays of data can be very

Graphical systems, visualization and multimediasccg.sk/~ferko/gsvm_2013_01motiv.pdf–raster graphics •Lunar Lander, Asteroids (1979) –vector graphics –vector displays 12

Datasheet - テクトロニクス · The 2460-EC Graphical Potentiostat brings speed, flexibility, and simplicity right to your fingertips. Its innovative graphical user interface

Graphical Displays for Meta-Analysis

Development of Improved Graphical Shawn St. Germain ... · Development of Improved Graphical Displays for an ... Primavera P6). Typically, ... The primary purpose of this pilot project

Graphical and Segmented Display Solutionsww1.microchip.com/downloads/en/DeviceDoc/00001699D.pdf · Graphical and Segmented Display Solutions 5 Graphical Displays Microchip offers

What Is the Value of Graphical Displays in Learning?

How does Statistics and Graphical Displayscseweb.ucsd.edu/~ricko/CSE3/Lie_with_Statistics-notes.pdf · If your goal is to lie, cheat, manipulate, or mislead, Graphical Displays are

VE-KNT Colour in Graphics and Visualisation Using colour on VDU displays for graphical presentations

Graphical displays for meta-analysis: An overview with ...psych.colorado.edu/~willcutt/pdfs/Anzures-Cabrera_2010.pdf · graphical displays for meta-analysis ... [18] or an ordering

6 Day 2: Graphical data displays and Exploratory Data Analysis · 6 Day 2: Graphical data displays and Exploratory Data Analysis A compelling reason to use R is for its graphics capabilities

Mr. Young's Math Website: Moses Brown School - Home...Questions on Topic 1: Graphical Displays Which of the following distributions are more likely to be skewed to the right than skewed

Www.panzea.org Panzea Graphical Map Viewer Tutorial With the Panzea graphical map viewer you can construct custom graphical displays of the map positions

Graphical Data Displays and Interpretation Wednesday, October 9

Chromera Speciation Software -- The Right Tool for the ...€¦ · experience, such as dictionary files, a data selector, and the ability to customize the graphical displays in a

Lecture 02. Graphical Displays Part 1

Interfacing Graphical Displays - AN0047 - Application Note

Frequency, Percentage, and Proportion Statistical Based Graphical Displays