Research Policies and Mechanisms: Key Points from

the National Mathematics Advisory Panel

Joan Ferrini-Mundy

Director, Division of Research on Learning in Formal and Informal Settings, National Science Foundation

National Math Panel Forum, October 7, 2008, Washington DC

• What standards were used for including research?

• What observations can be made about the state of educational research as it relates to mathematics education policy?

• First things first and continuous improvement; what can we do?

research about mathematics teaching and learning

research about mathematics teaching and learning

research about mathematics teaching and learning that could provide

evidence for policy making

research about mathematics teaching and learning

research about mathematics teaching and learning that could provide evidence for

policy making

research about mathematics teaching and learning that could provide evidence for policy making and meets standards of

evidence

Policy questions: cause and effect, inferences from samples to

populations

“Causal knowledge is essential to produce and to evaluate scientific research in crucial areas of national need, including mathematics education.” (p. 63)

research about mathematics teaching and learning that could provide evidence for policy making and meets standards of

evidence

“The Panel’s assertions and recommendations, therefore, are grounded in the highest quality evidence available from scientific studies.”

Place the strongest confidence in studies that:

• Test hypotheses

• Meet the highest methodological standards (internal validity)

• Have been replicated with diverse samples of students under conditions that warrant generalization (external validity)

Standards for studies of the effects of interventions

• High quality: random assignment to conditions, low attrition

• Moderate quality: matching statistical controls, or demonstration of baseline equivalence; low attrition or evidence of small attrition effects; valid and reliable measures

Where to find commentary about research

• Final Report:– Chapter 10, Research Policies and Mechanisms– Appendix C, Standards of Evidence

• Report of the Task Groups and Subcommittees– Chapter 2: Report of the subcommittee on

standards of evidence– Chapters 3-9, individual task group and

subcommittee reports

• The panel and contractors considered ~16,000 research publications and policy documents

• An unprecedented position from which to make comments about research about mathematics teaching and learning that could provide evidence for policy making

We need:

• “Sufficient supply of competent researchers dedicated to areas of critical national need”

• “Sufficient supply of willing schools and practitioners who have the time, resources, and motivation to be partners in research and use the findings of research in decision making”

• “A sufficient and stable source of funding for quality research and training with appropriate peer review”

Something for all:

• Mathematics education research community

• Researchers from a variety of fields• Leaders of graduate programs• State and district education policy

makers• Federal policy makers and funding

agencies

Mathematics education research communityand collaborators from other fields:

More research on the key questions that could provide evidence for policy making:

• “Effective instructional practices and materials• “Mechanisms of learning”• “Ways to enhance teachers’ effectiveness,

including teacher education that focuses on learning processes and outcomes”

• “Item and test features that improve the assessment of mathematical knowledge”

Leaders of graduate programs:

Redesign and improve programs and courses:

• “Attention to research design, analysis, and interpretation for teachers”

• “Increase national capacity to conduct and utilize rigorous research”

Researchers from a variety of fields

Pursue critical research issues in mathematics education:

• “Create cross disciplinary research teams”• “Include expertise from educational

psychology, sociology, economics, cognitive development, mathematics, and mathematics education”

State and district education policy makers

Collaborate in enabling educational research:• “Provide preK-12 schools with incentives and

resources to provide venues for, and encourage collaboration in, educational research”

• Follow progress of “resolutions to make individual student data available to researchers with appropriate safeguards for confidentiality”

A finding of the Instructional Practices group

“All encompassing recommendations that instruction should be entirely ‘student centered’ or ‘teacher directed’ are not supported by research.”

synthesize and theorize

evaluate and generalize

implement, study, and improve

design, develop, and test

hypothesize and clarify

An R&D Cycle for Improving K-12 Education:

Adapted from NSF/DRL solicitations, 2007-08

synthesize and theorize

evaluate and generalize

implement, study, and improve

design, develop, and test

hypothesize and clarify“to be ready for even small scale

experiments, basic research and intervention development studies are needed” (p. 63)

“smaller scale (less costly but highly informative) experiments (p. 63)

Large field trials that address problems of major national importance” (p. 63)

First things first and continuous improvement:

• Interpretation for policy makers and practitioners

• Scholarly discussion and critique

• Research agenda initiatives

• Research studies

• Capacity building

• New interdisciplinary collaborations