Private Schools & Student Achievement Private Schools & Student AchievementSlide 1 John Q. Scholar, HGSE, April, 2008 Does Attending a Private School Cause

Private Schools & Student AchievementPrivate Schools & Student AchievementSlide 1

John Q. Scholar, HGSE, April, 2008

Does Attending a Private Does Attending a Private School School CauseCause Children to Children to Have Higher Have Higher Achievement?Achievement?

John Q. ScholarJohn Q. ScholarCenter for Basic ResearchCenter for Basic Research

Harvard UniversityHarvard University

Graduate School of EducationGraduate School of Education

April, 2008April, 2008

Private Schools & Student AchievementPrivate Schools & Student Achievement Slide 2John Q. Scholar, HGSE, April, 2008

My Research Questions:My Research Questions:

1.1. Does receipt of a private school Does receipt of a private school tuition voucher lead families to tuition voucher lead families to place their children in private, place their children in private, rather than public, school ?rather than public, school ?

2.2. Does attending a private school Does attending a private school cause children to achieve at cause children to achieve at higher levels than children who higher levels than children who attend public schools?attend public schools?


Where Was My Study Conducted?Where Was My Study Conducted?

• In New York City, among low-income In New York City, among low-income families who had children about to enter families who had children about to enter kindergarten.kindergarten.

• Etc.Etc.

• Etc.Etc.


Who Is In My Sample?Who Is In My Sample?

• 1449 families drawn randomly from 1449 families drawn randomly from the population of 12,042 low-the population of 12,042 low-income families who volunteered to income families who volunteered to participate in the study.participate in the study.

– Summary stats on families?– Can only generalize results to

population of volunteers.

• Each family provided one target Each family provided one target child, for a total sample of 1449 child, for a total sample of 1449 children.children.

– Summary stats on children?– Statistical power analysis indicates

that this sample size provides moderate to high power (.80) to detect small effect sizes (.10 of a standard deviation) at typical levels of Type I error.


What Special Procedures Did I What Special Procedures Did I Employ:Employ:

• Administered Standardized Tests of Administered Standardized Tests of Reading Achievement “pre” and “post”:Reading Achievement “pre” and “post”:

– Pretest: at “baseline,” before the beginning of kindergarten.

– Post-test: at the end of the kindergarten year.

• Conducted A Randomized Experiment:Conducted A Randomized Experiment:– Families were randomized to either a

Treatment or a Control group.• Treatment families received a private-school

tuition voucher,• Control families did not.

– Families chose to enroll the target child in either a private or a public school, making up any difference in fees or expenses themselves.

– All target children were monitored through the end of their kindergarten year.


What Are My Measures:What Are My Measures:• Outcome VariableOutcome Variable::

– READ1READ1:: Nationally normed reading achievement, measured at the end of Kindergarten (expressed in national percentiles).

• Question Predictor:Question Predictor:– PRIVATEPRIVATE: : Dichotomous variable

indicating whethe the child attended a private school during kindergarten (1=Private; 0=Public).

• Instrument:Instrument:– VOUCHERVOUCHER: : Dichotomous variable

indicating whether the child’s family received a private school tuition voucher (1=Voucher; 0=No Voucher).

• Covariate:Covariate:– READ0READ0: : Nationally normed reading

achievement, measured at the beginning of Kindergarten(expressed in national percentiles).

0.0

50

.05

0 50 100

0

1

Rea

ding

Ach

ieve

men

t (%

ile)

read1Graphs by Revised 1st fu indicator for at

Public

Private

Mean = 24.19

Mean = 25.12


What Data-Analyses Did I Conduct:What Data-Analyses Did I Conduct:

RQ #2RQ #2: Does attending a private school cause children to : Does attending a private school cause children to achieve at higher levels than children who attend public achieve at higher levels than children who attend public schools?schools?

2nd stage of 2SLS/IV analysis. Multiple linear regression model, full sample.

Addressed my research questions by instrumental variable estimation (IVE), using a Two-Stage Least-Squares (2SLS) approach:

iiii READATEVPRIREAD 0ˆ1 210

iiii READVOUCHERPRIVATE 0210

RQ #1RQ #1: Does receipt of a private school tuition voucher lead : Does receipt of a private school tuition voucher lead families to place their children in private school?families to place their children in private school?

1st stage of 2SLS/IV analysis. Linear probability model, full sample.


Vouchers Impact Enrollment in Private Vouchers Impact Enrollment in Private SchoolSchool RQ #1RQ #1: Does receipt of a private school tuition voucher lead : Does receipt of a private school tuition voucher lead families to place their children in private school?families to place their children in private school?

ParameterParameterEstimateEstimate

(standard(standard

error)error)

Intercept, 0

.0447**(.0154)

VOUCHER, 1

.7710***(.0169)

READ0, 2

.0004(.0004)

2 .1021

R2 .59

Interpretation, as usual … being awarded a tuition

voucher increases the fitted probability that a family will

enroll its child in private school by 77 percentage

points … etc.

Interpretation, as usual … being awarded a tuition

voucher increases the fitted probability that a family will

enroll its child in private school by 77 percentage

points … etc.

Outcome = PRIVATE


Vouchers Impact Enrollment in Private Vouchers Impact Enrollment in Private SchoolSchool RQ #1RQ #1: Does receipt of a private school tuition voucher lead : Does receipt of a private school tuition voucher lead families to place their children in private school?families to place their children in private school?

0

1

0

10

100

200

300

400

500

600

700

Fre

Private

VoucherThe fitted odds that a family awarded a tuition voucher will

enroll its child in private school are 82.9 times the

odds that a family without a voucher will do the same.*

The fitted odds that a family awarded a tuition voucher will

enroll its child in private school are 82.9 times the

odds that a family without a voucher will do the same.*

* Setting Pretest reading achievement to the sample average of 22.54


Private & Public Schools Equally Private & Public Schools Equally EffectiveEffective RQ #2RQ #2: Does attending a private school cause children to : Does attending a private school cause children to achieve at higher levels than children who attend public achieve at higher levels than children who attend public schools?schools?

ParameterParameterEstimateEstimate

(standard(standard

error)error)

Intercept, β0

10.963***(.866)

PRIVATE, β1

.9061(1.185)

READ0, β2

.5875***(.0211)

2 299.3

R2 .35

Interpretation, as usual … cannot reject the null hypothesis that the effect of

private school is zero, but the associated coefficient is positive, suggesting that

enrolling in private school for the kindergarten year causes a child to score less than one percentile point higher on a

nationally-normed test of reading achievement.

Interpretation, as usual … cannot reject the null hypothesis that the effect of

private school is zero, but the associated coefficient is positive, suggesting that

enrolling in private school for the kindergarten year causes a child to score less than one percentile point higher on a

nationally-normed test of reading achievement.

Outcome = READ1


Interpretive VisualsInterpretive VisualsDisplaying RQ#2 FindingsDisplaying RQ#2 Findings

Private & Public Schools Equally Private & Public Schools Equally EffectiveEffective RQ #2RQ #2: Does attending a private school cause children to : Does attending a private school cause children to achieve at higher levels than children who attend public achieve at higher levels than children who attend public schools?schools?


Big Picture:Big Picture: Vouchers Affect Enrollment.Vouchers Affect Enrollment. The The

probability that a child will enroll in a probability that a child will enroll in a private school, for their kindergarten year, private school, for their kindergarten year, is 77 % points greater, if the family is is 77 % points greater, if the family is provided with a tuition voucher (p<.001). provided with a tuition voucher (p<.001).

Schools in Both Sectors Equally EffectiveSchools in Both Sectors Equally Effective. . A child who has attended private school A child who has attended private school for kindergarten scores less than 1 for kindergarten scores less than 1 percentile point higher on a nationally-percentile point higher on a nationally-normed test of reading, at the end of the normed test of reading, at the end of the year (p>.40).year (p>.40).


What Does This Mean for Kids What Does This Mean for Kids and Families?and Families?

Need to change state and federal Need to change state and federal educational policy by …educational policy by …

We do not need to buy shares in We do not need to buy shares in voucher printing companies.voucher printing companies.

Need to watch “The Bachelor: London Need to watch “The Bachelor: London Calling.”Calling.”

Etc.Etc.


Where Do We Go Next?Where Do We Go Next?In the future, we need to:In the future, we need to:• Distinguish the impact of vouchers on private school enrollment for important Distinguish the impact of vouchers on private school enrollment for important

subgroups of children:subgroups of children:– Race/ethnicity?– Gender?

• Examine larger samples of children to buttress the statistical power:Examine larger samples of children to buttress the statistical power:– Must ensure there is sufficient statistical power to detect effects in important

subgroups.

• Extend the study longitudinally, beyond the kindergarten year, through High Extend the study longitudinally, beyond the kindergarten year, through High School and into the workforce:School and into the workforce:

– Examine long term impact of both vouchers and private school on subsequent outcomes.

• Focus on other kinds of academic and non-academic outcomes:Focus on other kinds of academic and non-academic outcomes:– Mathematics, science knowledge, languages.– Measures of later school success (grade repetition, dropout, graduation, college

entry).– Social and affective outcomes (self-confidence, satisfaction, reliability,

character).– Economic outcomes (future labor force participation, wages). other than reading ,

and non-academic outcome

• Replicate the study in other major cities and states.Replicate the study in other major cities and states.

• Join the French Foreign LegionJoin the French Foreign Legion.


Include a BibliographyInclude a Bibliography(pay due homage -- (pay due homage --

Peterson, KruPeterson, Krueeger and ger and Angrist may be in the Angrist may be in the

audience!)audience!)


Appendices:Appendices: Include a data appendix.Include a data appendix.

Describe approaches you used to deal with Describe approaches you used to deal with missing data, and other analytic problems.missing data, and other analytic problems.

Provide ancillary descriptive statistics on Provide ancillary descriptive statistics on the full sample and important subgroups.the full sample and important subgroups.

Provide additional tables containing the Provide additional tables containing the results of fitting alternative model results of fitting alternative model specifications, sensitivity analyses, etc.specifications, sensitivity analyses, etc.

Include ancillary slides that anticipate Include ancillary slides that anticipate potential audience questions.potential audience questions.

Etc. …Etc. …


Overview of Main Points:Overview of Main Points:The The FourFour p’s … p’s …


PreparePrepare Your Presentation Your Presentation• Establish A Establish A Sensible Slide FormatSensible Slide Format That Is Common Across All Slides: That Is Common Across All Slides:

– Use footers and headers to maintain your title, authorship, affiliation, date.– Make sure each slide has an explicit title that explains what’s on that slide.– Don’t put too much on any one slide.

• Provide An Provide An Attention-Grabbing OpeningAttention-Grabbing Opening::– Slide #1: State title, author, affiliation, date, mention your personal interests.– Slide #2: State your research questions, discuss the literature and abiding issues.

• Document The Document The Background And Context Of Your ResearchBackground And Context Of Your Research::– Slide #3: Identify and describe your site.– Slide #4: Identify and describe your dataset.– Slide #5: Describe your sample, include selected descriptive statistics/plots, if

possible.– Slide #6: Describe any important procedures used in the design or data

collection.– Slide #7: Define your measures –outcome, question predictors, instruments,

covariates.– Slide #8: Overview your data-analyses, by question, identifying critical

parameters.

• Present Your Present Your Main FindingsMain Findings, By Research Question, By Research Question::– Slide #9a: Prepare decent, sparse, well-annotated summary tables, by RQ.– Slide #9b: Create interpretive visuals to accompany answer to each RQ, if possible.

• Provide A Provide A Memorable ClosingMemorable Closing::– Slide #10: Provide a “big picture” summary in words, directly addressing the RQs.– Slide #11: Comment on the “real-life” benefits or consequences of your work.– Slide #12: Outline the implications of your work for the future.

• Prepare Prepare AppendicesAppendices and Supplementary Slides (optional) and Supplementary Slides (optional)

• Prepare A Prepare A HandoutHandout For Your Audience, Including a Bibliography (optional, but a good idea) For Your Audience, Including a Bibliography (optional, but a good idea)..


PracticePractice Your Presentation Your Presentation• Review Your Visuals CarefullyReview Your Visuals Carefully::

– Examine their clarity, clutter and relevance.– Decide, in advance, which slide(s) you’ll skip, if time runs out.– Identify some additional slides you’ll comment on if time allows.

• Prepare To PresentPrepare To Present::– Plan every word you intend to say.– Write your script out in a large font, formatted to emulate spoken phrasing.– Use cautious animation to stage the presentation of each slide.– Avoid unnecessary animation (just like me?).

• Rehearse Your TalkRehearse Your Talk::– Videotape a practice run and review your performance with remote control

& coach.– Obtain feedback from your coach on:

• The strength of your opening & closing.• Your logical flow.• The credibility and clarity of evidence presentation.• The clarity and comprehensibility of your key points.• Your many distracting verbal mannerisms and physical ticks.

• Check the physical environment in which you’ll present, in advance:Check the physical environment in which you’ll present, in advance:– Check that you have everything you need (including a pointer).– Check the audience sightlines in advance, moving or removing seats as

needed.– Fix your computer so that it doesn’t go to sleep while you’re being

introduced.– Make sure that the projected image can be read from the back of the room.– Establish a reasonable surface to place (hide) your notes in plain sight.


Be PositiveBe Positive During Your During Your PresentationPresentation

• Be Positive About Be Positive About Your TalkYour Talk::– Project a sense that you really value your material.– Do NOT read a script, but give your talk using your slides as

“prompts.”

• Be Positive About Be Positive About YourselfYourself::– Be confident, yet relaxed.– Don’t be afraid to acknowledge a mistake, but don’t grovel.

• Build Build RapportRapport With Your Audience With Your Audience::– Be yourself, be sincere.– Establish and maintain eye-contact with your audience.– Connect your message to what you think their experiences are.– Don’t adopt a “defensive” position -- move out towards, and into,

the audience if that is possible.– If you don’t want questions until the end, say so!– If you don’t know the answer, don’t bluster – say “I don’t know

the answer to that right not, but I will certainly look into it!”

• Hold the Audience’s Hold the Audience’s AttentionAttention::– Don’t stand rigidly, gripping the podium with white knuckles.– Have good decisive body animation.– Don’t be afraid to direct audience questions back to other

audience members (or to the rest of the panel!).

Documents

Private Schools & Student Achievement Private Schools & Student AchievementSlide 1 John Q. Scholar, HGSE, April, 2008 Does Attending a Private School Cause