Aggregate Data

AGGREGATE DATA

AGENDA OBJECTIVES

Agenda and Objectives

• Descriptive statistics• Dispersion• Aggregate data• The right questions and graphics• Closing

Compare basic descriptive statistics and identify their limitations

Describe common mistakes associated with analyzing "on average" data

Explain the purpose of the Data Narrative analyses

Evaluate research questions against criteria for quality

In the last activity, we finished reviewing our

tree of statistical terminology.

Now let’s apply that knowledge!

Aggregated Data

Entertainers vs. Athletes for Class #1

• Who did better? How much better?• Should we be worried about the lower-performing group? Why, why not? • If you were the principal, would you intervene on behalf of the lower-

performing group? Is this teacher disfavoring entertainers?

Click ahead when you’ve completed the appropriate section

of your Handout

Check Your Work

• Athletes, on average, performed about 25 percentage points higher than entertainers

• That overall average difference is misleading. Except for Charlie Sheen vs. Serena Williams, everybody else in the two groups performed similarly

Statistical finding vs.

Interesting finding

Statistical finding vs.

Interesting findingAthletes performed better than

entertainers. But the difference was really just because Charlie Sheen

scored a 1 and Serena a 100.

Statistically significantvs.

Practically significantNot every statistical finding has any practical purpose. A number is just a number without any other context.

Speaking of…“Statistical significance”

Misuse of the Term “Statistically Significant”

The word "significant", in this sense, does not mean "large" or "important" as it does in the everyday use of the word.

http://xkcd.com/539/

Statistical significance

The Data Narrative is NOT a test of statistical significance! It’s an exploration of your data. It’s a report and analysis. It is NOT

statistical modeling.

Correct use of the Term “Statistically Significant”

• Statistically significant, in the statistical sense, refers to something that is unlikely to have occurred by chance. Like a scientific experiment performed in a laboratory setting.

CORRELATION CAUSATION

Unless you're using randomized trials and experimentation, in the statistical world, you cannot say that something caused something else. You can say that two

things are 'related', or may be 'contributing factors', but not they caused

each other.

Simpson’s Paradox:Why even an average of 1.5 years of growth is not necessarily good enough

Example #1: Longitudinal SAT Verbal scores

Newspaper headlines:Average scores don’t improve!

Overall

Average SAT Verbal 1981Average SAT Verbal 2002

Behind the scenes: Scores increases within every racial subgroup

Black/A

Hispan

ic/Lati

American

Indian

200240280320360400440480520560

Average Verbal SAT 1981Average Verbal SAT 2002

Describe the Paradoxical Nature of the Data

Group indexAverage SAT Verbal 1981

Average SAT Verbal 2002

White 519 527

Black/AfrAm 412 431

Asian 474 501

Hispanic/Latino 438 446

American Indian 471 479--------------------Overall average

-------------504

------------- 504

Click ahead when you’ve completed the appropriate section

of your Handout

Check Your Work – What’s The Paradox?

White 519 527

Black/AfrAm 412 431

Asian 474 501

-------------504

------------- 504

Scores Increase By Subgroup But Hold Constant Overall

White 519 527

Black/AfrAm 412 431

Asian 474 501

-------------504

------------- 504

Why the Paradox? Every subgroup increased their score.The percentage of test-takers in each group changed.

Test Takers 1981

White; 85%

Black/AfrAm; 9%

Asian; 3%

Hispanic/Latino; 2%

American Indian; 1%

Test Takers 2002

White; 65%

Black/AfrAm; 11%

Asian; 10%

Hispanic/Latino; 9%

American Indian; 1%

Other; 4%

“A statistician can have his head in an oven and his feet in ice…

and he will say that, on average,

and he will say that, on average, he feels fine.”

Be wary of “on

average”!

UC Berkeley: Was Admissions Biased?

• In 1973, a lawsuit was filed against UC Berkeley for discrimination – overall, 44% of men were admitted and only 35% of women were admitted to all graduate programs

http://hoopedia.nba.com/index.php?title=Oski_the_Bear_California

Nope, no bias. Women just applied to more competitive programs than men.

• In 1973, a lawsuit was filed against UC Berkeley for discrimination – overall, 44% of men were admitted and only 35% of women were admitted to all graduate programs

• It was discovered that, on average, women were applying to more competitive programs at higher rates (like law school, med school, etc), and therefore were being rejected with greater frequency overall across programs

GO BEARS!

Other Examples of Simpson’s Paradox

COMPARING BATTING AVERAGES: WHO LOOKS LIKE THE BETTER PLAYER?

Each year Justice is better, but overall Jeter is better.

Why the paradox?

1995 Average .250 Average .253

95/96 Average .310 Average .270

Other Examples of Simpson’s Paradox

COMPARING BATTING AVERAGES: WHO LOOKS LIKE THE BETTER PLAYER?

Player 1995 1996

Derek J Average .250 .314

Hits/AB 12/48 183/582

David J Average .253 .321

Hits/AB 104/411 45/140

Why the paradox?

95/96 Average .310 Average .270

What’s the Lesson from Simpson’s Paradox?

DISAGGREGATE YOUR DATA!(AND TELL THE RIGHT STORY)

Aggregate Data

Education

Review on Key-Aggregate Searchable Encryption (KASE) for ... · Review on Key-Aggregate Searchable Encryption (KASE) ... been proposed for data sharing via cloud storage, ... Aggregate

Aggregate Data Structures COS 441 Princeton University Fall 2004

Aggregate Data - United States Department of the Treasury · Aggregate Data HFA Performance Data Reporting - DPA Characteristics Q2 2016 Program Intake/Evaluation Assistance Characteristics

Lecture 9 Aggregate Data Organization

Oracle Data Integrator "Aggregate Function Transformation"

Introduction to aggregate data...Data about areas • known as aggregate data • counts of people, households, etc. with particular combinations of characteristics for an area •

Talent Management Data Mining - Discovering Gold in LAP 360 Aggregate Data

key aggregate cryptosystem for scalable data sharing in cloud

Building Technology Capabilities to Aggregate Clinical Data and

Aggregate Reporting and Data Disclosure Avoidance Techniques

Lecture 9 Aggregate Data Organization Topics Pointers Aggregate Data Array layout in memory Structures February 14, 2012 CSCE 212 Computer Architecture

Unit Record Versus Aggregate Data: Perspectives on …nces.ed.gov/pubs98/98280.pdf · 1998. 4. 8. · CHAPTER III NEW APPROACHES ... -- aggregate data methodologies -- is satisfactory

Optimizing Aggregate Query Processing in Cloud Data Warehousesxinwenfu/paper/Conferences/14_Globe... · 2 Optimizing Aggregate Query Processing in Cloud Data Warehouses (such as DHTs)

Model Specification, Use of Aggregate Data,hanushek.stanford.edu/sites/default/files/publications/Hanushek+Jac… · Model Specification, Use of Aggregate Data, and the Ecological

Provably Secure Key-Aggregate Cryptosystems with Broadcast … · Provably Secure Key-Aggregate Cryptosystems with Broadcast Aggregate Keys for Online Data Sharing on the Cloud Sikhar

Reference data sheet 1. Blast Furnace Slag – Aggregate

Lecture 4 Aggregate Data Types

Autonomous Aggregate Data Analytics in Untrusted Cloud

Data and the Aggregate Economy - Columbia …...This doesn't change rms' choices or rm dynamics. Does a ect aggregate quality and welfare. Laura Veldkamp (Columbia) Data and the Aggregate

Accessing UK census aggregate data using InFuse · Accessing UK census aggregate data using InFuse Richard Wiseman UK Data Service Census Applications: Using the UK population census