Upload
romeo
View
25
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Overview of Education Research & Development Centers Competition (84.305C). Rebecca McGill-Wilkinson, Ph.D . James Benson, Ph.D. Erin Higgins, Ph.D. & Jonathan Levy, Ph.D. National Center for Education Research. Organizational Structure of IES. Office of the Director. - PowerPoint PPT Presentation
Citation preview
ies.ed.gov
Connecting Research,Policy and Practice
Overview of Education Research & Development Centers Competition
(84.305C)
Rebecca McGill-Wilkinson, Ph.D.James Benson, Ph.D.
Erin Higgins, Ph.D. & Jonathan Levy, Ph.D.National Center for Education Research
ies.ed.gov
Organizational Structure of IES
2
National Board for Education
SciencesStandards &
Review Office
Office of the Director
National Center for Education Evaluation
National Center for Education Statistics
National Center for Education Research
National Center for
Special Education Research
ies.ed.gov
Missions of the Research Centers
• NCER– supports rigorous research that addresses the nation’s
most pressing education needs, from early childhood to adult education
• NCSER– sponsors a rigorous and comprehensive program of special
education research designed to expand the knowledge and understanding of infants, toddlers, and students with or at risk for disabilities from birth through high school
3
ies.ed.gov
IES Grant Programs: Research Objectives
• Develop or identify education interventions (practices, programs, policies, and approaches) – that enhance academic achievement– that can be widely deployed
• Identify what does not work and thereby encourage innovation and further research
• Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness
4
ies.ed.gov
Connecting Research,Policy and Practice
Education Research & Development Centers (84.305C)
ies.ed.gov 6
Research & Development Centers: Purpose
• Contribute to solving U.S. education problems, improve the education system, and increase student achievement through– Research – Development – Evaluation – National leadership
ies.ed.gov 7
Research & Development Centers: Activities
• Focused research program– Contribute to solution of specific education problem within
the Center’s topic– Generate new knowledge and theories
– Detailed plan of focused program required– 75% of activities should be for focused program of
research
ies.ed.gov 8
Research & Development Centers: Activities
• Supplemental activities– Meetings, smaller-scale studies– Work cooperatively with the Institute to select and design
– No detailed plan needed– Show capacity for conducting supplemental activities– Set aside up to 5% of budget annually
ies.ed.gov 9
Research & Development Centers: Activities
• Leadership and outreach– Engage with researchers, practitioners, and policymakers– Website with reports, papers, etc.– Innovative uses of technology (e.g., webinars, podcasts)
– Describe audiences and activities– Show capacity for leadership and outreach
ies.ed.gov 10
Connecting Research,Policy and Practice
Research & Development Center on Knowledge Utilization
ies.ed.gov 11
Purpose
• Research– Develop tools for observing and measuring research use in
schools– Illuminate the conditions under which practitioners use
researcher and factors that promote or inhibit research use in schools
– Identify strategies that make research more meaningful to and impactful on education practice
• Leadership and outreach– Demonstrate effective means of knowledge transfer to
improve school performance and student outcomes
ies.ed.gov 12
Background
• Findings from high-quality research and evaluation on impact of education policies and interventions
• Concern that this information is not used by practitioners– To adopt interventions with beneficial effects on students– To discard interventions with adverse or null effects
ies.ed.gov 13
Background: Potential Causes
• Misalignment between research questions (e.g., does it work?) and practitioners’ concerns (e.g., how do I use it with my students?)
• Misunderstanding of social context of practitioners– Research use is multilevel phenomenon (e.g. involves
districts, school leaders, and teachers)• Need new ways of working with practitioners– Partnerships– Bidirectional model of relationship– Intermediary organizations
ies.ed.gov 14
The Center’s Focused Program of Research
• One measurement study– Develop (or modify) and validate tools for observing and
measuring research use in schools and school districts• Two descriptive studies– Understand the conditions under which research is used
and factors that promote or inhibit research use in schools– Identify practices that are associated with greater use of
research in schools and school districts
ies.ed.gov 15
The Center’s Focused Program of Research
• Option to conduct Development Study or Efficacy Study– As alternative to one or both descriptive studies
• To develop a pilot intervention to improve research use in schools
• Or to conduct an efficacy study to assess the impact of a particular practice on the use of research in schools or school districts
• Study may range along a continuum of rigor
ies.ed.gov 16
Significance of Center
• Significance section at start of Research Narrative• Discuss the significance of your proposed Center
– Your vision of the Center – What problems it is intended to address– How your set of research and leadership activities will work
together to promote the use of scientific education research in schools and districts
• Discuss significance of three studies in focused research program
• Describe how you will engage researchers and practitioners in the work and communication of findings
ies.ed.gov 17
The Measurement Study: Research Questions
• What are the right indicators to know if school staff are using education research in their decision-making?
• How to capture variation in research use across schools, school district, and over time?
• You may expand or add additional research questions (and should provide justification for doing so)
ies.ed.gov 18
The Measurement Study: Methodological Requirements
• Methods of developing (or modifying) and validating assessment(s)
• Approach for gathering psychometric evidence• Sample and setting• Characteristics, size, and analytic adequacy of the
sample, including justification for inclusion and exclusion criteria
• Timeline
ies.ed.gov 19
Descriptive Study 1: Research Questions
• What role do State and local education agencies play in making research available to school administrators, faculty, and staff?
• What role do school leaders play?• How do education agencies, school leaders, and teachers obtain
and process scientific information?• How do social networks within schools and school districts appear
to support or hinder research use?• What are the major incentives or disincentives for practitioners to
better understand and apply education research?• What contextual factors seem to affect the use of research in
schools and school districts?• You may expand or add additional research questions (and should
provide justification for doing so)
ies.ed.gov 20
Descriptive Study 2: Research Questions
• Do certain communication strategies used by researchers or intermediaries correlate with greater use of research by practitioners?
• Is the presence or absence of intermediaries associated with greater use?
• Are particular technological innovations associated with greater use of research by practitioners?
• Are partnerships between researchers and practitioners associated with greater research use? To what extent do researchers perceive partnerships with practitioners as an enhancement or burden to their work?
• You may expand or add additional research questions (and should provide justification for doing so)
ies.ed.gov 21
Descriptive Studies: Methodological Requirements
• Population from which you’ll select your sample and sampling approach
• Measures• Data collection and analysis plan• Timeline
ies.ed.gov 22
Optional Development or Efficacy Study: Methodological Requirements
• Clear description of intervention to be developed or evaluated, including a theory of change
• Research questions• Sample and setting• Study designs may fall along a continuum of rigor but
efficacy studies should meet What Works Clearinghouse evidence standards
• Detailed power analysis
• Read Goal 2 and Goal 3 of 84.305A RFA for more details on methodological requirements
ies.ed.gov 23
Optional Development or Efficacy Study: Methodological Requirements
• Measures, including of fidelity and comparison group practices
• Key moderators and mediators• Plan to analyze implementation fidelity and
comparison group practices• Data analysis plan• Timeline
ies.ed.gov 24
Leadership & Outreach
• Innovative approaches to fostering dialogue among practitioners and researchers
• You may coordinate leadership/outreach with annual conferences to save time/money
• Institute encourages website• Institute open to learning how other applied social
sciences encourage research use among practitioners
ies.ed.gov 25
Maximum Awards
• 5 years• $5,000,000• Maximum of one award• Focused program of research must comprise at least
75% of the total budget
ies.ed.gov 26
Connecting Research,Policy and Practice
Research & Development Center on Standards in Schools
ies.ed.gov 27
Motivation
• Standards being rapidly deployed• >40 states currently implementing the Common Core State
Standards• Others implementing some form of college- and career-
ready standards• Little known about the actions that districts are taking to
implement the standards and the influence that the standards are having in schools and classrooms
ies.ed.gov 28
Motivation
• Need for a Network– To coordinate a research agenda on standards that
addresses the needs of practitioners and policymakers for timely information
– To coordinate efforts of researchers from multiple areas of expertise
ies.ed.gov 29
Purpose
• Timely and accurate information– The Center will be a source for unbiased information
regarding implementation of the standards. • Leadership, Outreach and Networking
– The Center will bring together researchers, practitioners, and policymakers who are actively engaged in standards-related research, practice, and policy development.
• Assess Early Outcomes of College- and Career-Ready standards – The Center will be at the forefront of assessing whether the
standards are making a difference, and for whom.
ies.ed.gov 30
Main Focus Areas for the Center
The Center must carry out three focused activities: 1. An Implementation Study2. A Longitudinal Outcomes Study 3. A Leadership and Outreach Program
The Center may also elect to conduct a Measurement Study.
ies.ed.gov 31
Implementation Study Goals
• How is implementation taking place?– What strategies are SEAs and/or LEAs taking to implement
college- and career-ready standards?– How do these strategies vary across sites? – How are implementation strategies seeking to address the
needs of key student groups including ELLs and students with disabilities?
– What changes, if any, are college- and career-ready standards bringing about in schools and classrooms?
ies.ed.gov 32
Implementation Study
• RFA encourages an expansive view of implementation, including several key dimensions, such as: – State, district and school leader support for implementation– Professional development for teachers – Curriculum materials– Teacher practice – Student assessment – All of the above as they pertain to ELLs and students with
disabilities – Broader contextual factors that may affect implementation
ies.ed.gov 33
Longitudinal Outcomes Study
• OPTIONS: Exploratory or Efficacy Study
• Twin goals 1. Assess the relation between standards implementation
and student outcomes2. Assess the factors that explain variation in student
performance across sites
ies.ed.gov 34
Possible Research Questions
• Should reflect the broad objectives for the Center
• As CCRS standards are implemented… – Do student outcomes improve or worsen as CCRS are
implemented, including for sub-groups? – Do achievement gaps between sub-groups improve or
worsen over the study period? – Which factors (implementation or contextual) moderate
the relation between the standards and student outcomes?
ies.ed.gov 35
Leadership & Outreach Activities
The Center should be a source of objective information for a variety of audiences from federal policymakers to local audiences.
• Minimum requirements – Provide specifics about the targeted audiences and
strategies to reach those audiences– Maintain a website– Convene researchers and stakeholders to discuss research
findings and needs
ies.ed.gov 36
Application Sections
• Significance of the Focused Program of Research• Research Plan for the Focused Program of Research• Plans for Other Center Activities (Leadership &
Outreach)
• Management and Institutional Resources• Personnel
ies.ed.gov 37
Application: Significance Section
• Opportunities and Challenges posed by college- and career-ready (CCR) standards– Major research issues in need of attention– How will your approach address these issues, and the
goals of the Center?• Conceptual Framework and Literature Review– What is your conceptual framework? – How is it grounded in theory on standards?– How will it guide research design, data collection and
analysis plans?
ies.ed.gov 38
Application: Significance Section
• Sites– Describe geography, school setting, student population,
implementation of standards– How will findings inform policy discussions and analysis? – Describe partnerships with SEAs, LEAs
• Letters of agreement (Appendix D)
• Research Agenda– Areas of focus for your Implementation Study– Research questions for your Outcomes Study– Justification for measures
• IF you propose measure development
ies.ed.gov 39
Application: Research Plan
Implementation Study– Sample selection, including population and sampling
method– Measures you will use– Data collection strategy– Analysis plan– Timeline
ies.ed.gov 40
Application: Research Plan
Longitudinal Outcomes Study• Exploration Study
– Population and sample selection– Measures you will use; student outcomes– Data collection and analysis plans– Timeline
• Efficacy Study – All of the above– Research design to produce impact estimates– Power analysis – Data analysis procedures– Dissemination plan
ies.ed.gov 41
Application: Research Plan
Optional Measurement Study• Provide a rationale for a new measure or substantial
modification to an existing measure– Instructional practice– Classroom activities– Opportunities to learn– Alignment of instruction, classroom activities, or curriculum
materials to college- and career-ready standards
ies.ed.gov 42
Application: Leadership and Outreach
• Target Audiences: Who will you be speaking to and what will be the key topics for each group?– National audiences (including federal policymakers and
national organizations)– Local audiences where the research will be conducted
• Role of practitioners in reviewing findings• Website (required) • Other Dissemination Venues – Products for downloading– Social media
ies.ed.gov 43
Maximum Awards
• 5 years• $10,000,000• Maximum of one award• Focused program of research must comprise at least
75% of the total budget
ies.ed.gov 44
Connecting Research,Policy and Practice
Research & Development Center on Virtual Learning
ies.ed.gov 45ies.ed.gov
Virtual Learning Lab (VLL): Background
• Many open questions:– Whether education technologies lead to meaningful
improvements in teaching and learning– How “big data” can be used to improve teaching and
learning
• Infrastructure is needed to:– Support the analytic and instructional use of the data– Store and share these data with others, and to mine data
generated from the use of education technologies
ies.ed.gov 46ies.ed.gov
VLL: Broad Purpose
• Evaluate and improve instructional practices, content, and/or learning tools offered by one or more widely used online instructional delivery platforms– Focus: Improving outcomes of students in K-12, particularly
low-income and low-performing students– One platform must be for K-12 students, but additional
platforms can be for either K-12 or postsecondary students
• Address how large amounts of data generated within these platforms can be used to improve teaching and learning
ies.ed.gov 47ies.ed.gov
VLL: Broad Purpose
• Inform stakeholders about the use of "big data" for education research and practice
• Create a “hub”
• Offer workshops and other activities
ies.ed.gov 48ies.ed.gov
Definition of “Widely Used Online Instructional Delivery Platform”
• The technological infrastructure used to: – Deliver any of a number of education technologies, and – Collect large amounts of information about students
• Platform(s) must be used across multiple states, in the majority of districts in a single state, or in the majority of schools in one or more large districts
ies.ed.gov 49ies.ed.gov
Definition of “Widely Used Online Instructional Delivery Platform”
• The online instructional delivery platform(s) you choose may be used to provide any of the following:– Online courses offered by a state, school district, or
postsecondary institution – Online courses with open enrollment (i.e., MOOCs)– Online schools or certificate/degree programs– Intelligent tutors– Blended learning models (combine online/classroom
instruction)– Flipped models (instruction online and homework/other
activities during class)
ies.ed.gov 50ies.ed.gov
Examples of Research Questions
• How can online and blended learning be used to address the instructional needs of learners across the spectrum – typical, advanced, and struggling – more effectively?
• How can online instruction be designed and implemented to promote the learning and retention of material, not just in the short term, but also over longer school-relevant time periods?
• How can online courses be designed so that students are more likely to persist in and complete the instruction?
• How can online instruction be targeted and delivered to meet the needs of low-income students, and to reduce achievement gaps between higher- and lower-performing students?
50
ies.ed.gov 51ies.ed.gov
Examples of Research Questions
• What data, metadata, analytical tools, and models are most useful for understanding and improving student learning?
• How can "big data" be used to predict students’ performance and help them receive the instruction and services they need?
• What practices, protocols, and policies should researchers and Institutional Review Boards (IRBs) be attentive to as they strive to protect students’ privacy?
• Which research and academic skill sets are needed to effectively conduct this type of work, and what training can be employed so others can develop the necessary skills?
ies.ed.gov 52ies.ed.gov
Important Partnerships
• The Institute strongly encourages partnerships between: – Researchers– Platform developers– Practitioners within states, school districts, and/or colleges
and universities who use online instructional delivery platforms.
• Must attach letters of agreement from all the key partners in Appendix D
ies.ed.gov 53ies.ed.gov
Project Narrative Requirements
• Significance of the Focused Program of Research• Research Plan for the Focused Program of Research• Leadership & Outreach Activities
ies.ed.gov 54ies.ed.gov
Significance of the Focused Program of Research
• Explain vision for the VLL• Identify problems or issues that the VLL will address– Review of research literature
– Provide data to support your argument (if possible)
• Discuss significance of studies proposed in the Research Plan– Explain how you will ensure generalizability of findings
ies.ed.gov 55ies.ed.gov
Significance of the Focused Program of Research
• Must identify at least ONE widely used online instructional delivery platform
• Platform must be intended for use by students in an authentic K-12 education setting – Additional platforms may be intended for use either by K-
12 or with postsecondary students
ies.ed.gov 56ies.ed.gov
Overview of Remainder of Presentation
• Components of the focused program of research– Experimental studies (min = 2)– Measurement studies (min = 1)
• "Big data"• Leadership and outreach activities
ies.ed.gov 57ies.ed.gov
Focused program of research:Experimental Studies
• At least TWO experimental studies– More can be proposed, depending on their scope
• Focus– Examination of strategies to improve instructional
practices, content, and/or learning tools within online delivery platforms
• Particular concern – Differentiation for individual learning needs
ies.ed.gov 58ies.ed.gov
What can be studied experimentally?
• Intervention that are…– Fully developed? – still in the development process?
• At least ONE study should be designed with the intention of improving an education technology that is in wide use
ies.ed.gov 59ies.ed.gov
What can be studied experimentally?
• Affordances in a digital environment– Easy, quick introduction and changes, based on findings– Support of timely decision making
• Comparison study of different iterations or versions of the intervention
ies.ed.gov 60ies.ed.gov
Measurement study (min = 1)
Goals• Explore/create valid and reliable measures• Determine which measures useful in differentiating
student learning needs• Identify mediating variables• Examine strategies to prepare data for user-friendly
consumption
ies.ed.gov 61ies.ed.gov
Research Plan: Experimental Studies
• Purpose: To help education technology developers and education practitioners make improvements in the instructional practices, content, and/or learning tools they are using
• See enumerated list (RFA, p. 26)
ies.ed.gov 62ies.ed.gov
Research Plan: Data Collection
Describe…• Sources of data
– Digital interactions– Administrative records– Surveys, Observations, and Interviews
• Methodology– Sampling strategy– Data collection plans– Analytic approach
• Rationale for each choice
ies.ed.gov 63ies.ed.gov
Studying a fully developed technology
• What are the critical questions?
• See enumerated list (RFA, p. 27)
ies.ed.gov 64ies.ed.gov
Research Plan: Measurement studies
• Purpose: What is the study’s justification?
• See enumerated list (RFA, p. 27)
ies.ed.gov 65ies.ed.gov
Developing and validating NEW measures
Describe iterative process…
• Field testing procedures & processes for item revision
• Validation activities, including types of evidence to be used for reliability and validation for the specified purpose, populations, and contexts
• See Measurement Goal section in the RFA 84.305A
ies.ed.gov 66ies.ed.gov
Concerns with "big data" and education
• Issues, concerns and implications on multiple levels– Practical, Ethical, and Legal
• Confidentiality, protection of PII – (personally identifiable information) from unauthorized
disclosure• Data access– Who should have access to data? For which purposes?
• Sharing, linking• Security rules and protocols
ies.ed.gov 67ies.ed.gov
Concerns with "big data" and education
• Negative side-effects of labeling – e.g., he’s a slow learner
• Unintended consequences
ies.ed.gov 68ies.ed.gov
Leadership & Outreach Activities
• Role of the VLL: Function as a reliable source• For Whom?– Technical: Researchers & developers– Broader audience: Teachers, administrators, policymakers,
general audience
ies.ed.gov 69ies.ed.gov
Dissemination
• What: Publications, products• How: Describe efforts planned efforts at
dissemination– Website (at a minimum)– Other new media (podcasts, video, webinars) that are
relatively inexpensive yet can reach a large audience
ies.ed.gov 70ies.ed.gov
Hub for Training & Research (some ideas)
• Researchers, developers, and practitioners will come together (virtually and in person)
• Discuss research goals and methods related to online learning
• Review emerging findings– Foster new research partnerships and collaborations:
(e.g., hosting non-Lab researchers, developers, etc., to work on Lab projects)
– Provide training and education: Host summer workshops, conduct courses for graduate students, non-Lab researchers, practitioners, and others
ies.ed.gov 71ies.ed.gov
Hub for Training & Research (some ideas)
• Organizing visits by state or school district leaders to demonstrate how education technologies are being used,
improved, and having an effect
• Your suggestion(s) here: ______
ies.ed.gov 72
Maximum Awards
• 5 years• $10,000,000• Maximum of one award• Focused program of research must comprise at least
75% of the total budget
ies.ed.gov 73
Connecting Research,Policy and Practice
General Requirements
ies.ed.gov 74
Additional Sections of the Research Narrative for All Centers
• In your 35-page Research Narrative, you are to discuss– Significance– Research Plan for the Focused Program of Research – Other Center Activities (Leadership and Outreach)– Management and Institutional Resources– Personnel
• We’ve discussed the first three and now we’ll move to the last two
ies.ed.gov 75
General Requirements: Management & Institutional Resources
• Describe plans and procedures for overall management of the Center and its activities
• Show capacity of supplemental activities and leadership and outreach
• If plan is to work in schools or other education delivery settings in first year, must include letter of agreement from education organization(s)
• If developer/distributer of program or intervention is part of Center, describe role and how objectivity will be maintained
ies.ed.gov 76
General Requirements: Personnel
• Expertise in content area(s)• Methodological expertise• Sufficient experience working with education
delivery settings• Capacity for supplemental activities• Experience relevant to national leadership– Explain why proposed Center staff is qualified to fulfill
leadership role
ies.ed.gov
Important Dates and Deadlines
77
Application Deadline
Letter of Intent Due
Date
Application Package Posted
Start Dates
August 7, 2014
4:30:00 PM DC Time
June 5, 2014 June 5, 2014 July 1, 2015to
Sept 1, 2015
ies.ed.gov
Peer Review Process
• Applications are reviewed for compliance and responsiveness to the RFA
• Applications that are compliant and responsive are assigned to a review panel
• Two or three panel members conduct a primary review of each application– Most competitive applications move to full panel
• At panel meeting, the most competitive applications are reviewed by full panel
78
ies.ed.gov
Notification
• All applicants will receive e-mail notification that the following information is available via the Applicant Notification System (ANS):• Status of award• Reviewer summary statement
• If you are not granted an award the first time, consider resubmitting and talking with your program officer
ies.ed.gov 80
Questions?
http://ies.ed.gov/funding
Knowledge UtilizationBecky McGill-Wilkinson ([email protected])
Standards in SchoolsJames Benson ([email protected])
Virtual LearningJonathan Levy ([email protected])
Erin Higgins ([email protected])