Mathematics and Science Partnership (MSP) Programs

Mathematics and Mathematics and Science Partnership Science Partnership

(MSP) Programs(MSP) Programs



U.S. Department of U.S. Department of Education: Education:

Administration Administration MeetingMeeting

The Annual Progress Report (APR)

• SEC. 2202.(f): Each eligible partnership receiving a grant or sub-grant shall report to the Secretary…

• Partnerships/projects must send their APRs to their State Educational Agency (SEA), the SEA must review and approve the APR, and submit it to the U.S. Department of Education (USED) within 60 days from the end of the performance period.

APR (continued…)• Typical Example:

– Project Achieve is awarded a 3 year MSP sub-grant that has a start date of 3/1/2009 and an end date of 2/28/2012 (performance cycle); with “performance period 1” starting on 3/1/2009 to 2/28/2010.

– The project has 12 months to complete their activities for that first year (as indicated in their application to the state).

– During the first 12 months the project would have completed their first year’s activities, collected student data, collected teacher data and would be ready to start the report by 2/28/2010.

APR (continued…)– The project finishes the APR and sends it to their SEA

(at a time determined by the SEA; before 60 days).– The SEA will review the APR and determine if

• The APR is complete and accurate; or• The APR needs revision.

– If the APR is complete and accurate the SEA will submit the report to USED by 4/30/2010 (60 days after the performance period).

– If the SEA has determined that the APR needs revisions, the SEA will comment on the report, the project will make the needed changes, resubmit to the SEA, the SEA will review again, and submit to USED by 4/30/2010 (60 days after the performance period).

APR (continued…)– This process will continue until…

• The project has completed a cycle (in this case 3 years) and submits their final APR (projects are encouraged to also submit their Final Evaluation Report that summarizes all 3 years of activities); or if

• The project is discontinued (a decision made by the SEA).

– At the end of the performance cycle projects must re-compete to receive an additional cycle of funding. (Note: not all states have 3 year cycles)

Overview of the APR Submission Process

(Norm)SEA creates an APR for the project to complete

The project completes the APR and submits it to the SEA by the indicated date.

The SEA reviews the APR and determines if the APR is acceptable OR if revisions are needed.

SEA sends the APR to the USED. The USED reports MSP results to Congress.

OR…Partial Submission (extraordinary)

SEA creates an APR for the project to complete

The project completes the APR and performs a partial submission and informs the SEA of when the APR will be completed by the indicated date. (Only with SEA approval).

The SEA reviews the partially submitted APR and determines if the APR is acceptable OR if revisions are needed & accepts or negotiates the final submission date with USED input.

SEA sends the APR to the USED. The USED reports MSP results to Congress.

Ed-MSP.NET

APR Due Dates

Accessing your APR• Post-award, your SEA coordinator should

create a new APR for newly funded projects and continuing projects.

• You should receive an email from Ed-MSP.net (check your spam box).

• If you have forgotten your login or password please access the forgotten password option on the login page located on http://apr.ed-msp.net.

Logging In

After accessing this link, it will ask you for your email (contact your SEA if you are not sure which email is listed).

The system will send you an email to re-establish your new login and password.

If that does not work contact [email protected] or your state’s contact at USED

mailto:[email protected]

Levels of AccessThere are three levels of access for projects

1. Project Directors can read, write, and submit reports to their state coordinators. (2 Max)

2. Other Contacts can read and edit reports. (5 Max)

3. Read-only access. (5 Max)*It is the project’s responsibility to tell the SEA

which individuals require access and their levels.

Completing the APR

Most of your answers can be found in the User's Guide…please refer to this document.

User’s Guide

A copy of the User’s Guide can be found as a part of your handouts!

User’s Guide Overview• Step-by-step instructions on how to

complete the report.• Includes definitions that describe what

each section should include.• Includes FAQ’s • Includes technical information on the

Mathematics and Science PartnershipsTeacher Content Knowledge (MSPTCK)

• User’s Guide for the MSPTCK

Completing the APR (continued…)

Lets the project see current APRs they must… -complete, or

-older APRs they have completed.

Browse Report…allows the project to enter data into the APR

Edit…allows the PD (only) to submit the report to the SEA coordinator.

Completing the APR (New Changes)!

In the Near Future, the Browse Report Link will be changed to “Edit”.

The Edit link will be changed to “Submit”.

Completing the APR (continued…)The Status column indicates where the project is in the APR submission process.

-Incomplete…not submitted to SEA

-Approval Pending…sent to SEA; SEA has yet to review.

-Revisions Needed…reviewed by the SEA; project must re-submit to the SEA after changes have been made.

-Partial Approval…project is missing core data (teacher, student, and/or evaluation); project must re-submit when data is complete by the approved date.

-Approval Granted…SEA has approved the APR and submitted to USED.

APR Table of Contents

APR Section I- MSP Project Information

This section provides a summary of basic project information:

a) Project: Name, funding amount, etc.b) Lead Organization: the fiscal agent.c) Partner Organizations: includes the fiscal

agent, the High-Need Local Education Agency (LEA), the Institute of Higher Education (IHE), partner schools (public and private), other LEAs (charters, etc.), businesses, non-profits, or other.

Section II- AbstractThis section asks the project to briefly

describe their project. The abstract should include a description of:-the participants (teachers, student, etc);-the professional development models and/or interventions;-any other information the project would like to share.

*The abstract will be viewed as a public document on the ed-The abstract will be viewed as a public document on the ed-msp web-site and msp web-site and must be cut & pasted into this sectionmust be cut & pasted into this section. It . It may notmay not be saved as an attachment. be saved as an attachment.

Section II- Abstract

Do not attach the abstract…please cut and paste or type into the text box below.

Section III- ResponsibilitiesThis section is designed to identify the various

partners’ roles and responsibilities.a) Administer the Overall Partnership: (logistical

management, budgeting, recruitment, etc).b) Designing Professional Development: (Creating PD

curriculum, designing learning tools, assessments, etc).

c) Deliver Professional Development: (Leading the PD sessions, technical assistance, etc).

d) Evaluate MSP: (Collecting and/or analyzing data, designing the evaluation plan, etc.).

Section III- Responsibilities

Projects should make their best estimate in this matter; however it should be noted that the percentages should add up to 100%

Section III- Responsibilities (continued…)

e) Number of Higher-Ed faculty.f) Indicate the Primary Target for intervention

1. Individual Teacher: (individual teacher model) intervention targets the individual teacher.

2. Schools: (teacher leader model) intervention targets teacher leaders (coaches); teacher leaders impact their schools’ student achievement level.

3. District: (teacher leader model) intervention targets teacher leaders (coaches); teacher leaders impact all student achievement within a district.

Section III- Responsibilities

Section IV- PD Participants

This section identifies the number of participants in an MSP project (especially the number of teachers served).

a) Elementary School Teachersb) Middle School Teachersc) High School Teachersd) Administratorse) Participant Students*Be sure to count each teacher only once.

Section IV- PD Participants

If a teacher teaches multiple groups of students (e.g. general education and AP) count the teacher only once – under the teacher’s primary/dominant role.

Section V- PD ModelsThis section asks the project to identify the

number of contact hours, the structural model, and a detailed account of activities in the professional development models.

a) Contact Hours: the average number of PD hours provided to each teacher (e.g. summer institute hours, on-line follow-up, etc…not independent study, etc.)

b) Type of PD Activities: this section asks the project to identify the structural model:

1. Summer institute: PD that is conducted for at least two weeks (80 hrs or a normal two-week time period).

2. Summer institute with follow-up: PD that is conducted for at least two weeks with follow-up in the school year.

3. Other: Any structure that is not listed above.c) Description of Professional Development Model

-Provide a description about the person that provided the PD and a more detailed account of the activities.

Section V- PD Models

Choosing a PD type will open a specific series of windows that allow the project to provide further detail.

Section VI- Professional Development Content

This section identifies the Professional Development Content.

• Mathematics Content• Science Content • Both

Section VI- Professional Development Content

Section VII- Program Evaluation

A) Evaluation Design: • A brief description of the evaluation design.• Did your project administer pre/post tests? When

did the tests take place (immediately before the PD, immediately after, 3 weeks after, etc.) & how often?

• Did you assess student content knowledge?• Did you use comparison groups? Were they

match comparisons, equivalent at baseline, etc. (please describe)?

• Did the project use other measures of success, if so please describe?

• Include any relevant information about sampling, data collection procedures, and analysis .


B) Design Type:• Who evaluated your project?

C) Evaluation Method:• Random Assignment• Matched Comparison• Non-Matched Comparison


D) Assessment Measures:• Provide information on eacheach assessment

used to evaluate teachers & information on each each assessment used to evaluate students.

• Indicate whether each assessment has been tested for validity and reliability.

*If it was developed for the MSP project, please indicate the sources for each of the questions and the number of questions for each source.


If the project uses multiple assessments (as many projects do)…the project must access this page and option multiple times to report on each assessment or evaluation tool.

Section VII- Program EvaluationE) Classroom Assessment:

• How did your project measure success in the classroom?

• Identify the protocol or instrument used to measure success in the classroom.

F) Phase of Implementation• Identify the project’s level of implementation

(this provides context for the USED to understand the available data).


Choose the Stage of Implementation

Section VII- Program EvaluationG) Impact on TeachersH) Impact on StudentsI) Other ImpactsJ) Impact on Partners*In these narrative sections, please

describe anecdotes or describe evidence of the program’s effectiveness.

Section VII- Program EvaluationK) Upload Report: Evaluation report

In the Evaluation Report, we are looking for specific information on how the evaluation was designed and implemented. • How did the project determine if teachers experienced

statistically significant gains in content knowledge?• Were there any threats of contamination? If so, how

were they handled?• Provide a timeline and description of each of the

evaluation activities.• The Final reports should address the Data Quality

Initiative (DQI) rubric criteria (if projects are reporting experimental or quasi-experimental designs).


If projects do not have the evaluation report ready by the date of submission, projects may (in consultation with the SEA and USED) partially submit the APR until the evaluation report is completed.


Please note…if this is your project’s final year (and will have to re-compete to acquire additional funds), please check this box & attach an evaluation that evaluated the project across its entire performance cycle.

This should include (among other things) the aggregate student and teacher information and evaluation across the life of the project.

Section VIII- Government Performance & Results Act

A) Teachers:• Use the MSPTCK spreadsheet to indicate the

number of teachers who have made statistically significant gains.

*Remember to save the spreadsheet to your personal computer as it will not save on the site.

B) Students:• Include student data using only the most important

assessment for mathematics or science.*For Both Sections:

-Do not combine information pertaining to different assessments for the same person; only use the most important assessment.-If the results are unavailable, projects may (with consultation of SEA and the USED) perform a partial submission.



Click the spreadsheet…

Note: if the project has administered more than one content knowledge assessment to different groups of teachers, run the MSPTCK for each group and aggregate the results.

If you have difficulty please access the users guide.


Select Enable Macros



If you need detailed instructions please access this link for step by step directions.


States may have changed their assessment structure, but the concept of application remains the same

Section IX- Lessons Learned

In this section we are asking the project to provide an overview of the challenges and successes encountered in the project over the past 12 months.A) MSP ImplementationB) MSP Evaluation

Section X- State Review

This section is to be filled out by the state coordinator.

A) AwardsB) Funding

Section X- Supplementary Docs.

Questions?• Patricia Johnson at

[email protected]• Miriam Lund at

[email protected]• Michelle Meier at

[email protected]• Jimmy Yun at [email protected]





U.S. Department of U.S. Department of Education: Education:

San Francisco Regional San Francisco Regional MeetingMeeting

Documents

Mathematics and Science Partnership (MSP) Programs