Authorizing Plan (AAP) Guidance. - MDE...Minnesota Statutes, section 124E.05, Subdivision 5, “The commissioner shall review an authorizer's performance every five years in a manner

March 30, 2017

Dr. Brad Blue Executive Director Minnesota Guild of Public Charter Schools PO Box 68152 Minneapolis, MN 55418

Dear Dr. Blue:

Thank you for your timely submission of evidence to address performance measures identified in the Minnesota Guild of Public Charter Schools’ (Guild) corrective action plan (CAP). The Minnesota Department of Education (MDE) recognizes your commitment to continuous improvement and dedication to authorizing excellence. Based on MDE’s review of evidence submitted for the Guild’s CAP, the Guild has satisfactorily addressed the performance measures identified in the CAP. The Guild’s performance rating is updated to “Satisfactory” and will be reflected on MDE’s website. A final rubric generated by MDE’s review of the Guild’s evidence submitted for the CAP is enclosed with this letter and provides complete information regarding MDE’s review.

The Guild now has the authority to charter new schools, accept transfers or initiate expansion requests. The Guild is also eligible to submit its next five-year plan. Please carefully review the attached Approved Authorizing Plan (AAP) Guidance. This document provides guidance to authorizers when updating their authorizing plans for the next five-year term which replaces the approved authorizer application (AAA).

Authorizers are expected to continue implementing their AAA until MDE accepts their AAP. We request the Guild’s AAP submission be submitted no later than May 1, 2017. In the event that additional time is needed or if you have any questions, please contact Paula Higgins, Charter Center Specialist, at 651-582-8315 or [email protected].

Sincerely,

Kevin McHenry Assistant Commissioner

CC: Jennifer R. Nelson, Director, Charter Center, MDE (email only) Philip Trobaugh, Charter Center Liaison, MDE (email only) Lisa Notermann, Charter Programs Coordinator, MDE (email only) Patrick Donnell, Federal Charter School Grant Manager, MDE (email only)

Encl. 17.03-30 Guild MAPES Corrective Action Review Rubric AAP Guidance

mailto:[email protected]


Updated 12/22/2016

Minnesota Authorizer Performance Evaluation System (MAPES) Introduction

The Minnesota Authorizer Performance Evaluation System (MAPES) was

established to review authorizer performance per Minnesota Statutes, section 124E.05, Subdivision 5, “The commissioner shall review an authorizer's performance every five years in a manner and form determined by the commissioner, subject to paragraphs (b) and (c), and may review an authorizer's performance more frequently at the commissioner's own initiative or at the request of a charter school operator, charter school board member, or other interested party. The commissioner, after completing the review, shall transmit a report with findings to the authorizer. (b) Consistent with this subdivision, the commissioner must: (1) use criteria appropriate to the authorizer and the schools it charters to review the authorizer's performance; and (2) consult with authorizers, charter school operators, and other charter school stakeholders in developing review criteria under this paragraph. (c) The commissioner's form must use existing department data on the authorizer to minimize duplicate reporting to the extent practicable. When reviewing an authorizer's performance under this subdivision, the commissioner must not: (1) fail to credit; (2) withhold points; or (3) otherwise penalize an authorizer for failing to charter additional schools or for the absence of complaints against the authorizer's current portfolio of charter schools.” i The development of MAPES was funded in part by an implementation grant from the National Association of Charter School Authorizers (NACSA) Fund for Authorizer Excellence. Through this grant, TeamWorks International was selected as the contractor to help MDE develop the initial plan and performance measures for MAPES. Development of MAPES

A main guiding principle in the development of MAPES was engaging

stakeholders. It was a priority of the Minnesota Department of Education (MDE) to involve stakeholders, particularly authorizers in the development of MAPES to create a fair and transparent evaluation system. The performance measures were initially presented to authorizers in January, 2014. Over the course of a year and a half, approximately 14 stakeholder engagement sessions were hosted by MDE. Stakeholders were engaged in multiple working groups and asked to share their thoughts on various versions of the performance measures in MAPES. All feedback provided to MDE was considered and substantial changes were made to MAPES based on stakeholder engagement. Some notable changes included minimizing the number of performance measures from 34 to 20 and changing the look back period from five years to three years. Performance measures were finalized and shared with authorizers in

https://www.revisor.mn.gov/statutes/?id=124E.05


Updated 12/22/2016

April, 2014 and specifications clarifying the indicators for the performance measures were sent to authorizers in January, 2015.

To promote an objective, uniform and consistent evaluation process for all four cohorts, a third party vendor was selected through a request for proposal process selected by peer reviewers. SchoolWorks, LLC. was selected as the contractor to facilitate and conduct all authorizer evaluation activities using MAPES performance measures and evaluation processes. Upon selection, each cohort was notified of the SchoolWorks evaluators involved with the evaluation process and asked to notify MDE if any actual or perceived conflict of interest existed. Authorizers were divided into four cohorts dependent on their initial approval as an authorizer. MAPES Methodology and Review Process

Authorizers were evaluated by nationally recognized standards and state

expectations for high quality authorizing, their established standards and processes stated in their most recently approved authorizer application (AAA) and how they applied standards and processes with fidelity across their portfolio of charter schools.

The performance measures for MAPES were divided into two parts: Part A – Capacity and Infrastructure and Part B – Processes and Decision-Making. Overall, Part A had 11 measures and accounted for 25% of an authorizer’s performance rating, and Part B had 9 measures and accounted for 75% of an authorizer’s performance rating. Please refer to MAPES Measures, Indicators and Specifications (Appendix A) for specific information about the performance measures used to evaluate authorizers.

Evaluation activities for a cohort occurred over a course of approximately five months. There were four phases of the evaluation. Phase One – Welcome and Data Collection included the welcome meeting, authorizer document/evidence submission and distribution of the charter school leadership survey. At the welcome meeting, a thorough walk-through was provided to authorizers on the purpose of the evaluation and the evaluation process outlined in the Review Process Summary (Appendix B) as well as MAPES Performance Rating and Outcomes (Appendix C). Authorizers were also provided with a USB drive for their document/evidence submission. A list of documents needed to meet at least satisfactory were provided to authorizers at the welcome meeting, but the list was also included in MAPES Measure, Indicators and Specifications (Appendix A). The charter school leadership survey was distributed to key school leadership personnel via email for each Local Education Agency (LEA) in an authorizer’s portfolio of charter schools (Appendix D). Survey questions aligned with MAPES performance measures that required external verification.

Phase Two – Data Review involved the evaluator reviewing existing authorizer data on file at MDE and new data submitted by the authorizer in Phase One. Existing MDE data included: approved authorizer applications; executed charter contracts; authorizer annual reports; MDE review data; authorizer income and expenditures reports; state portfolio performance data reports and other data maintained by MDE. Any data submitted to MDE prior to the MAPES Document Request Submission deadline was considered for evaluation purposes.

Updated 12/22/2016

Phase Three – Field Work was comprised of authorizer interviews and

stakeholder meetings. The purpose of the interview and meetings was to provide verification/validation of an authorizer’s established standards and processes outlined in their AAA and their implementation of those standards and processes. Interviews and meetings were conducted in-person or via Adobe Connect, a video conferencing platform. Please note that the interview and meetings did not serve as stand-alone evidence. All cohorts were provided with an opportunity to submit evidence after the interview.

Phase Four – Performance Reports included reviewing, finalizing and distributing MAPES reports. Prior to distributing to authorizers for review and comment, the evaluator provided a draft of the MAPES reports to MDE for review. The evaluator considered MDE feedback and provided a second draft to MDE for dissemination to authorizers for review and comment. While it was not possible for authorizers to submit new evidence at that time, authorizers had ten (10) business days to review draft report ratings, findings and key evidence and submit a response identifying and explaining any errors upon dissemination. This may have included information not accurately reflected in the report, previous evidence submitted that addressed a measure, but not included in the report, or information that misaligned with MAPES performance measures or with one or more data sources used in the evaluation. The evaluator considered all authorizer feedback and made corrections/updates based on errors of factual nature. Final reports were submitted to MDE and disseminated to authorizers no later than their five year term anniversary. Performance Ratings and Outcomes

The Authorizer Performance Ratings and Outcomes (Appendix C) outlined the next steps for authorizers depending on their overall rating. Authorizers received notification upon dissemination of their final MAPES report detailing next steps. Authorizers with rating outcomes of “Exemplary”, “Commendable” and “Satisfactory” were eligible to submit their next five year plan, also known as their Commissioner-Approved Authorizing Plans (AAP) to MDE.

Authorizers receiving an overall rating of “Unsatisfactory” or “Approaching Satisfactory” were ineligible to submit their AAP and were subjected to corrective action status. Per Minnesota States, section 124E.05, Subdivision 6, “The commissioner must notify the authorizer in writing of any findings that may subject the authorizer to corrective action and the authorizer then has 15 business days to request an informal hearing before the commissioner takes corrective action.” If an authorizer’s overall rating was updated to “Satisfactory”, the authorizer was then eligible to submit their AAP. If the informal hearing was waived by the authorizer or if the authorizer was placed in corrective action status after the informal hearing, an authorizer:

• Did not have the authority to charter new schools, accept transfers or initiate

expansion requests while in corrective action and until an overall


Updated 12/22/2016

performance rating of “Satisfactory “ was attained (i.e. out of corrective action status).

• Had up to one year to satisfactorily address all performance measures to be eligible to submit their AAP.

• Was only evaluated on deficient measures during the corrective action period.

An authorizer’s corrective action status was assessed on a case by case basis,

depending on the nature and scope of deficiencies. If identified deficiencies remained unaddressed, termination of an authorizer’s approval to charter schools occurred per Minnesota Statues, section 124E.05 Subdivision 6(a). Continuous Improvement

It is important to note for the purpose of continuous improvement, minor aspects of MAPES were made after cohort one and after cohort two evaluations; however, the fundamental aspects of MAPES (i.e. performance measures and evaluation processes) remained unchanged throughout all four cohorts to promote consistent evaluations.

The following changes were made during cohort one evaluations: • Strictly continuous improvement measures A.5, A.9, A.10, B.7 and B8 were

updated to reflect relative work within the last 12 months and not the past 12 months.

• Measures A.11 and B.3 were updated as both included charter contract compliance; contract compliance was taken out of A.11 and remained in B.3.

• The heading for B.6 was updated to Authorizer’s Standards and Processes for Interventions, Corrective Action and Response to Complaints.

• The guiding question for B.8 was updated to read, “To what degree does the authorizer plan and promote, within its portfolio, the model replication and dissemination of best practices of high performing charter schools.” This revision clarified that replication and best practices did not have to originate from an authorizer’s portfolio, but could come from other sources (e.g. had a clear plan to seek high performing models in other states, or encouraged adaption of high quality models or practices from other schools with the schools in an authorizer’s portfolio).

• The heading for B.8 was updated to High Quality Charter School Replication and Dissemination of Best School Practices.

The following changes were made starting with cohort two evaluations: • MDE data for A.11 and B.3 was disseminated to authorizers prior to the

evaluators’ data review, ensuring accurate submission records were used for MAPES.

• A MDE representative was included at the authorizer and school leader interviews and served an as observer


Updated 12/22/2016

• The Review Process Summary document provided to authorizers at the welcome meeting and MAPES Performance Ratings and Outcomes document were updated to clarify outcomes of MAPES.

• It was clarified to authorizers they had up to a week from their interview date to provide additional documentation for MAPES.

The following changes were made starting with cohort three evaluations: • The corrective action process was clarified at the welcome meeting and in

related documents. Conclusion

MDE commends and appreciates authorizer engagement in the MAPES process and dedication to improving authorizer excellence. MDE enjoyed learning about authorizer oversight work to promote high quality education options for all students in Minnesota. Similar to the development of MAPES, MDE recognized the importance of engaging authorizers and stakeholders in refining MAPES upon completing all four cohort evaluations in December, 2016.

Attached is Minnesota Guild of Public Charter Schools’ MAPES performance

rating and report (Attachment A) issued December, 2016. Please contact Paula Higgins, Charter Center Specialist, at [email protected] or 651 582-8315 with questions regarding MAPES.

i This document has been updated to include statutory changes from the 2016 legislative session.


Attachment A: Final MAPES

Performance Report- Minnesota Guild of

Public Charter Schools

Minnesota Guild of Public Charter Schools – MAPES Report Fall 2016 1

Minnesota Guild of Public Charter Schools – Minnesota Authorizers Performance Evaluation System (MAPES)

Authorizer Information

Authorizer: Minnesota Guild of Public Charter Schools (the Guild)

Authorizer Type: Single-purpose

MDE approved: Fall 2011

Operational Schools: Art and Science Academy, Augsburg Fairview Academy, College Prep Elementary, Community School of Excellence, LIFE Prep (previously Concordia Creative Learning Academy), Lincoln International High School, Metropolitan Education for Future Employment, New Heights School, Rochester Beacon Academy, Rochester Off-Campus High School, T.R.U.T.H. Preparatory Academy Charter School

Schools approved, but not open: FUSE (formerly Learning ReImagined), Marine Area Community School, Mercy Community School, Minnesota Academy for Hearing, Speech and Language, Peter Seeger Renaissance Charter School, Phoenix Academy Charter School, Skyline Math and Science Academy

Closed Schools: Mill City High School, TEAL Elementary Academy for Learning

Characteristics of the Authorizer

The Minnesota Guild of Public Charter Schools (the Guild) is a single purpose authorizer of charter schools that was conceived as a project of the Minneapolis Federation of Teachers, Local 59, AFT, AFLCIO to authorize charter schools that are dedicated to Albert Shanker’s vision of charter schools

as laboratories of innovation in which legally empowered educators are able to unleash creativity that will result in fresh, new ideas intended to revitalize all public schools.

The Guild espouses teacher leadership and actively supports teacher-powered schools. The Guild’s mission is to grow student achievement by authorizing quality charter schools and

monitoring their performance for sustainable impact. The Guild is governed by a six-member, volunteer Executive Board of Directors. A Director and Director

of Projects hold primary responsibility for operations and oversight of the portfolio of schools while supplementary responsibilities are assigned to Guild Advisors and Contractors.

Performance Rating Drivers

The authorizer did not provide evidence that the mission stated in the AAA reflects what they are realizing as an authorizer; in addition, the school leaders were unable to verify the authorizer’s

mission. The authorizer did not provide evidence that they have rigorous selection criteria to promote high

quality charter schools. No schools, new or transfer, have yet to be denied for approval. It is unclear how the authorizer makes decisions around resource allocation. A review of the Guild 5

Year Budget Projection did not include accurate dates, provide details regarding projected revenue assumptions, nor reflect a financial reserve should the authorizer not meet its projections. Insufficient evidence was provided to show that the authorizer has sufficient resources to effectively oversee its portfolio of charter schools.


The authorizer did not provide evidence that it consistently monitors and oversees the charter schools in the areas of academics, operations, and finances according to the processes outlined in the contract and approved authorizer application.

Overall Performance Rating

The MAPES Overall Performance Rating for the Minneapolis Guild of Public Charter Schools is 1.15: Approaching Satisfactory

Ratings Summary

PERFORMANCE MEASURES A: AUTHORIZER CAPACITY AND INFRASTRUCTURE – 25% Weight of Overall Rating Authorizer Mission and Vision

A.1: Authorizer Mission (5%) 1 A.2: Authorizer Vision and Organizational Goals (10%) 1

Authorizer Capacity and Infrastructure

A.3: Authorizer Structure of Operations (15%) 1 A.4: Authorizer Staff Expertise (10%)* 1 A.5: Authorizer Capacity and Skill Development of Leadership and Authorizing Staff (5%)* 3 A.6: Authorizer Operational Budget for Authorizing the Portfolio of Charter Schools (10%) 1 A.7: Authorizer Operational Conflicts of Interest (10%) 3 A.8: Ensuring Autonomy of the Schools in the Portfolio (15%) 4 A.9: Authorizer Self-Evaluation of Capacity, Infrastructure and Practices (5%)* 1 A.10: Authorizer High Quality Authorizing Dissemination (5%)* 0 A.11: Authorizer Compliance to Responsibilities Stated in Statute (10%) 3 Total Performance Measures A Rating: 1.9

PERFORMANCE MEASURES B: AUTHORIZER PROCESSES AND DECISION MAKING – 75% Weight of Overall Rating Authorizer Process and Decision-making

B.1: New Charter School Decisions (20% / 5%)** 1 B.2: Interim Accountability Decisions (10% / 5%)** 1

Authorizer Performance Contracting

B.3: Contract Term, Negotiation, and Execution (10%) 3 B.4: Performance Standards (10%) 1

Authorizer Ongoing Oversight and Evaluation

B.5: Authorizer’s Processes for Ongoing Oversight of the Portfolio of Charter Schools (10%) 1 B.6: Authorizer’s Standards and Processes for Interventions, Corrective Action and

Response to Complaints (10%)* 0 B.7: Charter School Support, Development and Technical Assistance (5%)* 1 B.8: High Quality Charter School Replication and/or Dissemination of Best School

Practices (5%)* 1 Authorizer Renewal and Decision-making

B.9: Charter School Renewal or Termination Decision (20%) 0


Total Performance Measures B Rating: 0.9

*Continuous Improvement Measure

**Weights adjusted for authorizers not engaged in B.1 and/or B.2 activities


Evidence to Support the Performance Rating

Using MAPES, the review team considered documentation for the Minnesota Guild of Public Charter Schools (the Guild). Following the review of the authorizer’s evidence of mission, capacity and infrastructure, process

and decision-making, performance contracting, ongoing oversight and evaluation, and renewal decision-making, the team determined consensus findings and attributed evidence to justify them (below). These findings form the basis for the team’s MAPES performance rating.

PERFORMANCE MEASURES A: AUTHORIZER CAPACITY AND INFRASTRUCTURE Authorizer Mission and Vision

A.1 - Does the authorizer have a clear and compelling mission for charter school authorizing?

Performance Level Rating: Level 1: Approaching Satisfactory

Finding: The authorizer has a clear and compelling mission for charter school authorizing; however, the authorizer is not implementing the mission included in the Approved Authorizer Application (AAA). The mission included in the authorizer’s AAA states, “The Minnesota Guild of Public Charter Schools [the Guild] will grow student achievement and success by authorizing quality charter schools and monitoring their performance for sustainable impact.” This mission aligns with Minnesota statute §124E.05, Subd. 4(1). For example, the Operating Guide states that the Guild will authorize and monitor charter schools in accordance with Minnesota statute §124.D.10 [recoded as 124E.05.D.4], which governs the duties and responsibilities of charter school authorizers. Further, the authorizer narrative states that the mission is aligned with Minnesota statute §124E: to improve all pupil learning and all student achievement and Governor Dayton’s 7-Point Plan for Excellence in Education. However, although this is the mission that was included in the A.1 Narrative, a review of the authorizer’s

website shows that the posted mission is different from the AAA. It states the mission as, “As a single-purpose authorizer, the Guild will improve the educational achievement of Minnesota students by advocating a systemic approach to charter school formation and operation that promotes stakeholder accountability and competence.” This mission was also evident in a review of board meeting minutes. During the authorizer interview, participants could not verify the mission, stating that the mission is about “holding schools accountable for the performance

measures in the contract,” and “[providing] teachers the opportunity to have a professional voice in their work.” In addition, the school leaders did not verify the authorizer’s mission, stating the mission is to provide oversight and direction to ensure high quality with a focus on the teacher powered model. The authorizer did not provide evidence that the mission stated in the AAA reflects what they are realizing as an authorizer. While the Guild has approved four new (and currently operational) schools, two new (to open in 2017) schools, is drafting five contracts for new schools, and has approved transfer of sponsorship of seven schools, the authorizer did not provide documentation of how they are ensuring the quality of their charter schools or how they have advocated for a “systemic formation and operation” of schools to

improve student performance.


Key Evidence:

A.1 Narrative, p 1 Minnesota Guild of Public Charter Schools Application (Approved Authorizer Application), p. 1 Minnesota Guild of Public Charter Schools Operating Guide, p. 4 Guild Board Agenda_08_29.2016 Guild Schools Website Home Guild Schools – Update 2.3 (08.21.2016) Authorizer Interview – October 11, 2016 School Director Interview – October 12, 2016

A.2 - Does the authorizer have a comprehensive vision for charter school authorizing with clear organizational goals and timeframes for achievement that are aligned with the purposes of Minnesota Law?


Finding: Although the authorizer has a comprehensive vision for charter school authorizing that aligns with the purposes of the Minnesota law, the authorizer is not implementing the vision from the AAA. According to the AAA, the Guild’s vision is that it will be “a national model for charter school authorizers

by advancing the original vision of the charter school model, in which teachers and parents look to research-based models and teacher expertise to identify collaborative innovations that promote effective learning, organization and management.” The AAA also outlines clear goals that are directly

aligned with the chartering vision and state statute. These include: improving pupil learning and student achievement; increasing learning opportunities for pupils; encouraging the use of different and innovative teaching methods, including measuring learning outcomes and creating different innovative methods of measuring outcomes; establishing new forms of accountability for schools and creating new professional opportunities for teachers, including the opportunity to be responsible for the learning program at the school site. These goals are also included in contracts between the authorizer and chartered schools. Yet, neither the AAA nor additional documentation outlines clear timeframes by which the organizational goals will be met. Further, although a copy of a 2014 NACSA evaluation was shared, no evidence was provided to demonstrate that the authorizer engages in self-evaluation of work against the chartering vision and progress toward organizational goals. During the authorizer interview, participants stated that the intention is for the Guild to have authorized “35 schools in 5

years,” but the authorizer did not articulate a specific timeline by which organizational milestones would be met. The authorizer does not implement the vision from the AAA. According to the authorizer’s website and

copies of board meeting minutes, the authorizer’s current vision is: “The Guild advances the original

vision of the chartered-school model, in which teachers are professionally organized and work to create innovative, research-based schools that rely on teacher expertise to identify and use effective teaching strategies, promote engaged student learning, create professional autonomy, and ensure effective, shared responsibility for outcomes.” During the authorizer interview, participants confirmed that this revised vision was currently guiding the organization’s work.


Key Evidence:

A.2 Narrative, p. 1 Minnesota Guild of Public Charter Schools Application (Approved Authorizer Application), p. 1 Guild Board Agenda_08_29.2016 AFA 12.18.15 Guild Schools Website Vision Sample contract NACSA Authorizer Evaluation Report (December 8, 2014) Authorizer Interview – October 11, 2016

PERFORMANCE MEASURES A: AUTHORIZER CAPACITY AND INFRASTRUCTURE

Authorizer Capacity and Infrastructure

A.3 - To what degree does the authorizer operate with a clear structure of duties and responsibilities and sufficient resources to effectively oversee its portfolio of charter schools?


Finding: While the authorizer generally operates with a clear structure of duties and responsibilities, it does not have sufficient resources to effectively oversee its portfolio of charter schools. The structure of duties and responsibilities is clearly defined. A review of the Guild’s organizational chart shows that the

authorizing organization is clearly defined. The organization is governed by an Executive Board that is, according to the A.3 Narrative, all volunteer. According to this organizational chart, the Director reports directly to the Executive Board, while the Director of Projects and contracted advisors report to the Director. The narrative states that “workflow is divided between four groups…: Guild Staff [Director and

Director of Projects], Guild Executive Board, Guild Advisors and Contractors.” Job descriptions for authorizer staff (Director, Director of Projects, and Advisors) outline the structure of duties and responsibilities. During the authorizer interview, participants explained that the role of the Director of Projects is largely to manage documents and track technical details, including oversight of Epicenter, the organization’s document management system. They also explained that, in addition to working with the Treasurer to ensure that the organization is fiscally sound, the Director ensures that the organization is in compliance with applicable state laws, that contracts are executed following the requirements set forth by the Minnesota Department of Education (MDE), and that the authorizer has adequate staff to monitor the schools within their portfolio. In the Charter School Leadership Survey, five (5) respondents agreed or strongly agreed that the structure of duties and responsibilities of the authorizer is clear and at a level adequate to meet the need of the school, while one (1) participant disagreed (n=6). However, during the School Director Interview, some participants articulated that they do not believe that the Guild has staffing at a level sufficient to meet the needs of the portfolio. As one participant summarized, “We have found that they need additional staff. I think they have a lack of capacity there. They have a lot of schools and some get lost in the shuffle. Especially as we were opening, we felt there was not enough support for us.” It should be noted that, according to the narrative and organization chart, the position of Director of Projects was added in March 2015 to address staff capacity at the Guild.


The authorizer appropriately manages and safeguards school, student information, and records relating to authorizing. According to the A.3 Narrative, and confirmed during the authorizer interview, the Guild uses Epicenter, a cloud-based platform, to manage school information. The narrative states that all documents are securely uploaded by schools. It explains, “The information is delivered through

encrypted communication channels to a secure data center with nightly backups and enterprise-class anti-virus software.”

Key Evidence:

A.3 Narrative GuildOrgChart_2016 Guild Advisor – Job Description Guild Director – Job Description Guild Director of Projects – Job Description Charter School Leadership Survey, question K Authorizer Interview – October 11, 2016 School Director Interview – October 12, 2016

A.4 - To what degree does the authorizer have appropriate experience, expertise, and skills to sufficiently oversee the portfolio of charter schools?


Finding: The authorizer has limited experience, expertise, and skills to sufficiently oversee the portfolio of charter schools. The authorizing staff has experience in charter schools, curriculum instruction, and management, but limited experience in law, facilities, and finance. A review of the resume for the authorizer’s Director shows that he has served as a director of two different charter schools since 2006. He holds licenses in a number of areas, including (but not limited to) K-12 principal. Although his resume reflects extensive educational experience, it does not reflect expertise (including experience and licensures) in finance, facilities, and law. A review of the Director of Projects resume shows that she worked in a charter school between 2012-2015, when she was hired by the Guild. She has management experience, but her resume does not reflect expertise in the areas of facilities, finance and law. The authorizer submitted resumes for eight (8) advisors with whom it contracts. Although some of these advisors listed in their resumes that they have attended trainings on school finance, there is no indication that any of them have expertise in this area. Further, although one board member was elected to serve in the Minnesota House of Representatives on the House Education Finance committee, the authorizer did not provide evidence to show that he has expertise and skills in finance. In email correspondence provided by the authorizer, some advisors reported having experience around facilities, including oversight of the relocation of a charter school and negotiating facilities to expand two sites. One advisor (Carrie Bakken) holds a law degree, though it is unclear the extent to which she has applied this education to her current work. Finally, a review of six executive board member resumes shows that one (Alessandra Vitrella) has law experience, having attended law school and served as legal clerk from 2010-2011; however, it is not clear that her work experience and skills provide expertise in school law.

Key Evidence:

A.4 Narrative


Advisor Bakken Resume Advisor Houtz Resume Advisor Kundan Resume Board Vitrella Resume Staff Blue Resume Staff Koppes Resume

A.5 - To what degree does the authorizer build the knowledge and skill base of its authorizing leadership and staff through professional development? Is professional development aligned with its operations, vision and goals for overseeing its portfolio of charter schools?

Performance Level Rating: Level 3: Commendable

Finding: The authorizer builds the knowledge and skill base of its authorizing staff through professional development that is aligned with its operations, vision, and goals for overseeing its portfolio of charter schools. Professional development is offered regularly to authorizing leadership and staff, and staff members attend professional development more than once a year. In the A.5 Narrative, the authorizer describes the professional development attended by both the organization’s Director and Director of

Projects. For example, since 2011, the Guild’s Director has participated in, or attended, twelve (12) trainings or conferences aligned with authorizing practices or policy work, and has presented in a number of other education venues. Since her start with the Guild in March 2015, the Director of Projects has participated in at least twenty-three (23) trainings or webinars focused on authorizing expectations, Epicenter, and legislative updates.

Professional development is differentiated for staff and aligns with the operations, vision, and goals for overseeing its portfolio of charter schools. The A.5 Narrative outlines the different professional development for the Director and Director of Projects. A review of this list shows that each staff member attends different trainings based on her/his role and responsibility. For example, the Director has regularly attended the National Association of Charter School Authorizer’s (NACSA) annual conference,

Minnesota Association of Charter School Authorizer (MACSA) monthly meetings, and conferences such as FinnFest (2014) and the International Summit on the Teaching Professional (2012), all of which align with the goals and vision of the Guild. In 2015, the Director attended a Conference on Teacher-Powered Schools, which aligns directly with the authorizer’s vision. The Director of Project’s trainings

have been largely focused on Epicenter, new charter school trainings, and required webinars through MDE. The authorizer did not submit evidence to show that professional development is measured and evaluated. When asked during the authorizer interview, participants could not articulate how they measure and evaluate the impact of their own participation in trainings and conferences.

Key Evidence:

A.5 Narrative Gmail – Registration Confirmation_FinnFest Gmail – Teacher Powered School National Conference Session Sign ups MACSA Meetings Authorizer Interview – October 11, 2016


A.6 - To what degree is the authorizer’s actual resource allocation commensurate with its

stated budget, needs and responsibilities of authorizing the portfolio of charter schools?


Finding: Resource allocations for authorizing are at least consistent with resources committed in the AAA; however, it is unclear whether current resource allocations are sufficient. A review of the 5-year projected budget included in the AAA generally aligns with the current resource allocations of the authorizer. The budget within the AAA makes assumptions for a gradual increase of the portfolio of authorized schools, beginning with two schools in FY 2012 and up to sixteen schools by FY 2015. It also makes an assumption that, based on this growth, the authorizer would add a part-time staff member in addition to the full-time Director. While the authorizer’s portfolio had not directly followed the

projections listed in the AAA budget, it currently has eight operational schools and three that are developing. According to the A.6 Narrative, the authorizer added a Director of Projects in March 2015 to support the increase of schools within the portfolio. The authorizer provided no evidence that resource allocations are devoted to achieving nationally recognized quality standards for authorizing.

It is not clear, however, whether current resource allocations are sustainable. When asked for annual (and adjusted) budgets to reflect changes in projections and to explain revenue assumptions, the authorizer did not submit these. Rather, they submitted financials for the authorizing organization (the Guild). Therefore, it is unclear how the authorizer makes decisions around resources allocations. Further, a review of the Guild 5 Year Budget Projection submitted by the authorizer did not include accurate dates, provide details regarding projected revenue assumptions, nor reflect a financial reserve should the authorizer not meet its projections. Moreover, in the A.6 Narrative, the authorizer states, “The Guild receives a great deal of consulting work both from the executive board and contractors pro

bono. This allows the Guild to be a viable business entity while striving to help schools do transformational work with their students rather than worrying about finances.” It is unclear whether the

authorizer’s projected financial model depends on pro bono services indefinitely, and the implications should these services be charged for in the future. During the authorizer interview, participants stated that the size of the portfolio is critical to the budget’s bottom line. They stated, “If we were to stay at the

current staffing level and not grow our portfolio, we would need a 28% or more increase in authorizing fee.” Additionally, they stated that, without legislative changes that would allow them to raise their fees, the current revenues for the organization are not sufficient.

Key Evidence:

Minnesota Guild of Public Charter Schools Attachments (Approved Authorizer Application), Attachment A, p. 73

Guild Form 990 2012 Guild Form 990 2013 Guild Form 990 2014 Guild 5 Year Budget Projection 2013/2014 Income and Expenditure Report 2014/2015 Income and Expenditure Report Authorizer Interview – October 11, 2016 Authorizer Interview – October 24, 2016


A.7 - To what degree does the authorizer implement a clear policy to address conflicts of interest in all decision making processes concerning the portfolio of charter schools?


Finding: The authorizer has a clear policy to address conflicts of interest in all decision making processes concerning the portfolio of charter schools. The Guild has included a clear conflict of interest policy in its Operating Guide, which states that it “applies to Guild staff, Advisory Board members, and Executive Board members.” The authorizer avoids conflicts of interest that might affect its capacity to make objective, merit-based applications. For example, the Application Review Form for each school contains the conflict of interest policy and asks whether a conflict of interest exists with the school or school founders. (It should be noted that the authorizer had not yet engaged in the renewal process with schools.) During the authorizer interview, participants explained that the conflict of interest policy is implemented with both the executive board members and with advisors working directly with schools. The authorizer provided one fully documented example of how they have been able to successfully implement their conflict of interest policy. In the narrative, the authorizer explained that one of its advisors assigned to the Metropolitan Education for Future Employment (MEFE), during which time she attended board meetings, acted as the Guild’s liaison and conducted annual site visits. When this

advisor was then hired as MEFE’s Executive Director, she then resigned her position as Guild Advisor to MEFE, ensuring that work with the relationship between the authorizer and the school was free of conflicts of interest. This resignation was documented in a review of board meeting minutes from July 15, 2016. When school leaders were asked about how the authorizer avoids conflicts of interest, some participants provided examples to show how the authorizer has implemented the conflict of interest policy. As one participant explained, “The few times I have asked questions that have pushed the boundaries, [the Director] has been clear about what he can and can’t do.” However, other participants articulated that conflicts of interest have been “fuzzy.” For example, one participant articulated that, although the authorizer explicitly stated that they did not want to “push their agenda” on the school, they

were also told that they would not be authorized unless they agreed to have unionized teachers. It should be noted that several members of the authorizer executive board have close affiliations with different teachers’ unions, including the Vice Chair of the Board of Directors who currently serves as the President of the Minneapolis Federation of the Teachers.

Key Evidence:

A.4 Narrative A.7 Narrative Minnesota Guild of Public Charter Schools Operating Guide, p. 22 Guild Application Review Form Authorizer Interview – October 11, 2016 School Director Interview – October 12, 2016

A.8 - To what degree does the authorizer preserve and support the essential autonomies of the portfolio of charter schools?

Performance Level Rating: Level 4: Exemplary

Finding: The authorizer preserves and supports the essential autonomies of the portfolio of charter schools. The authorizer has a clear policy to ensure school autonomy. When asked about the authorizer’s school autonomy policy, the authorizer explained that it is built into the contract. The


authorizer’s policy on school autonomy establishes and recognizes the school’s authority over

academic, operational and financial need and respects the school’s authority over the school’s day-to-day operations. A review of the contract showed that section 2.2 states that the authorizer “shall have

no authority or control over operational, administrative, or financial responsibility for the school.” In

section 6.1, the contract states that the school’s board shall ultimately exert control over the school,

including in matters related to operations, instruction, teacher contracts, and the budget process. The authorizer’s policy aligns with nationally recognized principles and standards for quality. According to NACSA, a quality authorizer preserves the following core autonomies: school governing board independence from the authorizer; personnel; school vision and culture; instructional programming; and budgeting. Based on its contract with schools and confirmation from school representatives, the authorizer’s policy is aligned to these essential autonomies. Moreover, according to school representatives, the authorizer’s practices align with its stated policy to uphold school autonomy. During the School Director Interview, participants agreed that the authorizer has granted schools autonomy. They cited several examples showing that the Director provides oversight and data, but is not overly involved in the processes and operations of the school. As one participant summarized, “the authorizer’s role is not to manage or direct in a certain way, unless there is an issue. [The Director’s]

approach is more hands off. But then, when you need something or answers, he is quick to respond. He provides information and resources.”

Key Evidence:

A.8 Narrative 14.10.08 Contract Guild and RBA FINAL 2014, p. 4 14.02-25 Executed Contract – Guild for ASA CORRECTED, pp. 2, 4-5 NACSA Principles_Standards, p. 8 Authorizer Interview – October 11, 2016 School Director Interview – October 12, 2016

A.9 - To what degree does the authorizer self-evaluate its internal ability (capacity, infrastructure, and practices) to oversee the portfolio of charter schools?


Finding: The authorizer does not regularly self-evaluate its internal ability (capacity, infrastructure, and practices) to oversee the portfolio of charter schools. The authorizer may have an informal review of its internal ability to oversee its portfolio; however, these self-evaluations are neither intentional nor planned to build the authorizer’s capacity, infrastructure and practices. According to a spreadsheet tracking Guild schools, the authorizer contracted with its first school in 2014. In 2015, it added four schools to its portfolio, either as new schools or transfers. In 2016, it added 6 new schools to its portfolio, all but one as transfer schools. It plans to authorize up to eleven additional schools by 2018. In March 2015, the authorizer hired a Director of Projects to share some of the added responsibility associated with the expansion of the portfolio and increased expectations from MDE. During the authorizer interview, the authorizer explained that it participated in a NACSA evaluation aligned to quality authorizer practices. However, this review, conducted in 2014, focused largely on application and contracting processes and policies; further, the authorizer did not provide evidence that an intentional or planned self-evaluation occurred within the last 12 months or that it regularly reviews its internal capacity, infrastructure and practices to oversee its expanding portfolio of charter schools.


Key Evidence:

A.9 Narrative Guild Schools – Update 2.3 (08.21.2016 – MAPES) AE_MN Guild_Draft Intro Ltr and Guide NACSA Authorizer Evaluation Report (December 8, 2014) Authorizer Interview – October 11, 2016

A.10 - To what degree does the authorizer disseminate best authorizing practices and/or assist other authorizers in high quality authorizing?

Performance Level Rating: Level 0: Unsatisfactory or Incomplete

Finding: The authorizer does not disseminate best authorizing practices and/or assist other authorizers in high quality authorizing. Best practices are rarely shared with authorizers. In the A.10 Narrative, the authorizer explains that it has shared best practices through its regular attendance at MACSA meetings; however, a review of minutes from meetings between January 2015 and May 2016 shows that, while the authorizer was in regular attendance at all meetings, there is no documented evidence that the Director shared best practices or assisted other authorizers around their authoring practices. The authorizer provided evidence that they have reached out to other authorizers for guidance around their own authorizing practices; but they did not submit evidence to show that other authorizers have contacted the Guild to learn about their practices. Moreover, although the authorizer submitted brochures from annual NACSA conferences, there is no indication within those brochures that the authorizer served as a speaker. In the narrative and during the authorizer interview, participants explained that they had been instrumental in helping to organize travel among a group of authorizers to learn more about Epicenter and authorizer compliance; however, documents submitted by the authorizer following the interview were insufficient to confirm this. Rather, they included an email (not from the authorizer) and a roster of participants.

Key Evidence:

A.10 Narrative MACSA Meetings Gmail – Friends of Education Complaint Process Gmail – Pillsbury_Guild CMO Requirements 2011 NACSA Conference Brochure_FINAL 2013 NACSA Conference Brochure_FINAL 2014 NACSA Conference Brochure_FINAL Gmail – RE_Michigan Trip – Participant Roster AuthorizerTripRoster Authorizer Interview – October 11, 2016

A.11 - To what degree does the authorizer comply with reporting, submissions, and deadlines set forth in Minnesota Statute?


Finding: The authorizer has consistently been in compliance with reporting, submissions and deadlines set forth in the Minnesota Statutes over the last three years. According to the documentation provided


by MDE, the authorizer submitted its income and expenditures reports on or before the due date for FY 2014, FY 2015, and FY 2016. The Guild did not submit its income and expenditures report on, or before, the due date for FY 2013. Therefore, because the report for FY 2013 was not in statutory compliance, the authorizer was compliant for the last three, but not four, years in all stated areas. The Guild did submit its Authorizer Annual Report on January 15, 2015, the day it was due; MDE confirmed its statutory compliance. Finally, at least one member of the Guild staff participated in the required annual report training on November 7, 2014.

Key Evidence:

A.11 Narrative 16.10-12 Cohort 4- Measures A.11 and B.3 Data - FINAL

PERFORMANCE MEASURES B: AUTHORIZER PROCESSES AND DECISION MAKING Authorizer Process and Decision Making

B.1 - To what degree does the authorizer have clear and comprehensive approval criteria and process standards to rigorously evaluate new charter school proposals? To what degree did the authorizer’s decisions and resulting actions align to its stated approval and process standards and promote the growth of high quality charter schools?


Finding: The authorizer does not have clear and comprehensive approval criteria and process standards to evaluate new charter school proposals. Additionally, it is unclear to what degree the authorizer’s decisions and resulting actions have led to the promotion of high quality charter schools. The authorizer’s application processes are comprehensive, and include clear application questions and guidance. In the B.1 Narrative and the Operating Guide, the authorizer outlines the steps of the application process. This includes a letter of intent to apply at the onset of the process to determine initially whether the applicant is a “good fit” with the Guild’s mission. If the authorizer is interested in

receiving a complete application, it provides feedback and the applicant then submits a complete application. The authorizer also has a new school application document that outlines requirements for each component of the application, including clear questions that must be answered by the applicant in areas that include (but are not limited to): statutory purpose; mission and vision; teaching or learning program; goals and student performance; evaluation plan; governance plan; administration plan; and operations plan. However, the authorizer’s process does not include transparent procedures. Although the charter application for new schools explains that applications will be scored on a scale of inadequate, satisfactory or excellent, the authorizer does not define what criteria determine each of these ratings. While the authorizer submitted an example of an advisor review for Marine Area Community School, in none of the documents (including the scored rubric, the Operating Guide or New School Application) are decision-making criteria outlined. The bases for the advisor ratings within the review are unclear.

The authorizer does not have a clear decision-making process that promotes the growth of high quality charter schools. The application template states that an application that is scored satisfactory in each rating will be sent to the authorizer’s Executive Board, which has 30 days to act on the

recommendation; however, criteria used by the Executive Board to make a final decision are not


defined. As stated above, the authorizer did not include a rubric or other guidance documents to define what criteria would lead to a satisfactory rating (versus excellent). Moreover, when asked during the interview whether they had formally denied a new school application for any reason, the authorizer stated that they had not. Rather, they have provided feedback to applicants and, in some instances, helped them revise their application for resubmission and approval. The authorizer did not submit documentation of the feedback provided to applicants. Thus, the authorizer did not provide evidence that they have rigorous selection criteria to promote high quality charter schools.

Key Evidence:

B.1 Narrative Minnesota Guild of Public Charter Schools Operating Guide, pp. 27-28 Minnesota Guild of Public Charter Schools Application New Schools Advisor Review – Marine Area Community School Application Marine Area Community School Guild Flow Chart 5-10-2016 Marine Approval Letter from MDE Authorizer Interview – October 11, 2016

B.2 - To what degree does the authorizer have clear and comprehensive approval criteria and process standards to rigorously evaluate proposals of existing charter school expansion requests and other interim changes? To what degree do the authorizer’s decisions and resulting actions regarding charter school expansion and other interim changes align to its stated approval and process standards and promote the growth of high-quality charter schools?


Finding: The authorizer does not have clear and comprehensive approval criteria and process standards to rigorously evaluate proposals of existing charter school expansion requests and other interim changes. Additionally, the authorizer does not have a clear decision-making process that promotes the growth of high quality charter schools. The authorizer’s application process is comprehensive, and includes clear application questions and guidance. Within the application for existing schools, the authorizer outlines the process that schools must follow to transfer authorizer. This includes the completion of an Intent to Apply and, if initially approved by the authorizer, a full application. The full application requires applicants to complete sections regarding the purpose fulfilled by the charter school, its vision and mission, its goals and student performance expectations, the education program model, the evaluation plan and applicant capacity, its governance plan, the financial management plan, and plans for both administration and operations. The authorizer’s application

process does not, however, include transparent procedures. Although the existing school application includes a scoring guide at the bottom of each section with ratings of inadequate, satisfactory and excellent, the authorizer does not include a rubric or other guidance document to define the scoring criteria. While the authorizer submitted an example of an advisor review for the Augsburg Fairview Academy and the Lincoln International High School, in none of the documents (including the scored rubric, the Operating Guide or New School Application) are decision-making criteria outlined. The bases for the advisor ratings within the review are unclear.


The authorizer submitted a copy of its School Readiness Plan, a comprehensive checklist to ensure that approved schools are ready to open. Additionally, the authorizer submitted documentation from Rochester Off-Campus Charter High School (ROC) to show the transfer review process, including a report submitted by the school, school board meeting minutes during which introductions were made between the Guild and the school’s board, the letter of intent submitted by the school, the school’s

application for transfer, and a copy of the authorizer’s review of the application. The authorizer did not

submit documentation to show the process by which the Guild made final determination that the school would be approved for transfer.

The authorizer’s interim accountability decisions are consistent with its criteria as stated in the AAA.

The authorizer’s AAA states that the Guild will consider expansions per Minnesota statute 124D.10.

Subd. 4(j) and the narrative states that expansion application requests are guided by MDE supplemental affidavit instructions. At the time of review, the authorizer was engaged in its first and only expansion request (specifically, an enrollment increase and the addition of a site to accommodate the increase). The authorizer provided a copy of the Art and Science Academy (ASA) Supplemental Affidavit (sent to MDE on October 1, 2016), demonstrating that the Guild reviewed the expansion request and made a final determination to approve the expansion request. Review of the affidavit shows that the school seeking expansion had not met academic performance goals; however, a description of the steps to address low performance were provided. The AAA states that in order to expand, expansion must be warranted, at a minimum, by longitudinal data demonstrating students' improved academic performance and growth on statewide assessments under chapter 120B. A review of ASA’s Supplemental Affidavit shows that ASA demonstrated growth in mathematics proficiency between 2015 and 2016 for grades 3-5, thus fulfilling the expectations laid out in the AAA.

The authorizer does not have a clear decision-making process that promotes the growth of high quality charter schools. The application template states that an application that is scored satisfactory in each rating will be sent to the authorizer’s Executive Board, which has 30 days to act on the

recommendation; however, criteria used by the Executive Board to make a final decision are not defined. As stated above, the authorizer did not include a rubric or other guidance documents to define what criteria would lead to a satisfactory rating (versus excellent). Moreover, when asked during the interview whether they had formally denied a transfer school application, the authorizer stated that they had not. Thus, the authorizer did not provide evidence that they have rigorous selection criteria to promote high quality charter schools.

Key Evidence:

B.2 Narrative FY16_15.12-08 Supplemental Affidavit Instructions FINAL (from MDE) ASA Supplemental Affidavit (10.03.2016) Guild – Existing School Application Guild Schools Supplemental Application FY2016 RCTC Report to MN Guild ROC Board Minutes 12.17.2014.pdf ROC Transfer Application Review ROC Minnesota Guild of Public Charter Schools app ROC Intent to Transfer Letter MACS School Readiness Plan


NACSA Principles_Standards Minnesota Guild of Public Charter Schools Operating Guide 4.25.16 Guild Board Meeting Minutes.pdf

PERFORMANCE MEASURES B: AUTHORIZER PROCESSES AND DECISION MAKING Authorizer Performance Contracting

B.3 - To what degree does the authorizer execute contracts that clearly define material terms and rights and responsibilities of the school and the authorizer?


Finding: The authorizer executes contracts that clearly define the material terms and rights and responsibilities of the school and the authorizer. The contracts in the authorizer’s portfolio of charter

schools meet current statutory requirements. According to MDE records, all the authorizer’s contracts

(six of which are for new schools, seven of which are change of authorizer) were submitted within 10 business days of execution. In some instances, the authorizer provided email correspondence from the MDE confirming that submitted contracts aligned with the statute.

The contracts clearly state the rights and responsibilities of the school and the authorizer. Within all of the contracts, Article II outlines the relationships between the school and authorizer. This includes sub-sections that articulate the following: voluntary authorization; independent states of the school; financial obligations are separate; no authority to obligate or bind other party; limited use of the authorizer’s

name. Article III specifically describes the role of the authorizer, including the oversight responsibilities and the authorizer fee.

Authorizer practices are consistent across the authorizer’s portfolio of charter schools. A review of the contracts within the portfolio confirmed that the authorizer uses a standard template off of which to build each contract with schools, leading to consistency across its portfolio. A review of communications between Community School of Excellence (CSE) and the authorizer exemplified the contract negotiation process. During the School Director Interview, participants described similar contracting practices with the authorizer, including a meeting during which they created goals around academic achievement, attendance, finance, governance, compliance, and the school’s governing board.

Additionally, the authorizer defines material terms within the contract. This was confirmed by school directors during the interview. The authorizer approved its first school in 2014 and has met specifications for at least the last two years.

Key Evidence:

B.3 Narrative CSE-Guild exchange 06.03.2016 CSE-Guild exchange 06.07.2016.pdf CSE, Recall please read 06.07.2016 Guild CSE Affidavit Part C Guild-CSE_Legal Expenses 12.18.2015 GuildResonseCSE_05.20.2016 MDE approval of CSE CIA 04.27.2016


MDE Charter Contract Guidance (01.28.216)-FINAL.pdf Minnesota Statutes 124E(2015)

B.4 - To what degree does the authorizer execute contracts with clear, measurable, and attainable performance standards?


Finding: The contracts define clear, measurable and attainable performance standards. Contracts in the authorizer’s portfolio of charter schools meet current statutory performance standards. A review of the contracts indicates that each contract outlines a description of the school program and the specific academic and non-academic outcomes that pupils must achieve in accordance with Minnesota statue

§124.E.10. Subd. 1. For example, the contract between the authorizer and T.R.U.T.H. Preparatory Academy lists ten (10) performance goals that include growth goals in reading and math, proficiency goals in reading and math, a proficiency goal in STEM, and goals around increasing the graduation rate, attendance, and parent/guardian satisfaction. Additionally, the contract includes financial and organizational performance measures. These performance indicators are included in the annual site visit report template included as an exhibit within each contract. In the Charter School Leadership Survey, three (3) respondents strongly agreed that the contract has clear, measurable performance standards. As one respondent explained, “I am confident that the performance standards are rigorous

yet attainable. They also address the unique needs of our school.” One respondent disagreed,

explaining that they were not aware of performance standards within the contract.

A review of the AAA indicates that contracts do not align with the performance standards of the AAA. The AAA states that “contracts will include language per school and authorizer agreement that school

operation shall be measured by the school performance indicators set forth in the contract including academic outcomes for individual students and for the school as a whole, & standards for governance, financial management & school operation.” Like T.R.U.T.H. Preparatory Academy’s contract, the

contract with Community School of Excellence (CSE) outlines ten achievement goals by which the authorizer will monitor student academic performance; however, neither contract includes academic outcomes for individual students.

Key Evidence:

B.4 Narrative Guild_CSE Contract (07.01.16) 15.11.-03 Guild-TRUTH Contract_signed NACSA Core Performance Framework and Guidance Minnesota Guild of Public Charter Schools Application (Approved Authorizer Application), p. 20 Charter School Leadership Survey, questions AO and AP


PERFORMANCE MEASURES B: AUTHORIZER PROCESSES AND DECISION MAKING Authorizer Ongoing Oversight and Evaluation

B.5 - To what degree does the authorizer monitor and oversee the charter schools in the areas of academics, operations, and finances according to the processes outlined in the contract and approved authorizer application?


Finding: The authorizer does not consistently monitor and oversee the charter schools in the areas of academics, operations, and finances according to the processes outlined in the contract and approved authorizer application. The Guild’s AAA includes clear processes for oversight and monitoring. According to the Operating Guide, the authorizer will initiate “an ongoing system of oversight to monitor and review the accountabilities the school must fulfill to meet its performance goal and commitments” in

the areas of governance, student/school performance, fiscal performance, and operations. The AAA outlines a number of actions they will take to monitor the school’s performance, including (but not

limited to): attending at least two school board meetings annually; reviewing the School Annual Report to check compliance with the contract agreement; reviewing the MDE report card to measure academic progress; reviewing financial statements on a monthly basis, the annual audit, and annual budgets as needed; and conducting an annual site visit.

The authorizer provided examples of its monitoring and evaluation practices for one school (MEFE). The majority of the operational schools within the authorizer’s portfolio were approved and/or opened in

2015 or 2016. A review of contracts with all of its schools shows that the authorizer includes consistent language around the monitoring and oversight of student performance. Moreover, in addition to being included in the Operating Guide, all contracts between the authorizer and its schools include the Annual Site Visit Report template that is utilized to measure compliance in alignment with the contract.

However, the authorizer’s oversight practices do not fully align with its stated oversight and monitoring processes outlined in the AAA. A review of the Metropolitan Education for Future Employment (MEFE) Annual Site Visit form for FY16 confirmed that the authorizer evaluated the school according to performance measures outlined in the contract agreement. Additionally, the authorizer submitted evidence that a representative attended the school’s governing board meeting on a monthly basis

between November 2015 and July 2016. The MEFE Board Meeting Observations spreadsheet notes that “financial reports include register, balance sheet, rev/exp, and cash flow,” indicating that the

authorizer received financial statements on a monthly basis. Yet, the authorizer did not submit evidence that it collected or reviewed any of its schools’ annual reports to check compliance against the contract agreement. In the B.5 Narrative, the authorizer stated that it utilizes Epicenter to oversee school compliance; however, the authorizer did not submit any documented evidence that they are indeed collecting and evaluating documents for compliance purposes. Further, the authorizer did not present evidence to show that it reviewed annual audits for any of its schools or monitored academic progress. It should be noted that MEFE is a new school that began operating in 2015 and may not have an existing audit. This, however, was not addressed by the authorizer. Although the authorizer conducted a site visit to MEFE on October 1, 2015, the site visit report provided limited evidence aligned to academics and finances. The three one-sentence notes from the site visit centered on touring the school’s facilities, the hiring of a new school director, and human resource items (employment,

insurance, and background checks).


Key Evidence:

B.5 Narrative Minnesota Guild of Public Charter Schools Operating Guide, pp. 31, 77-82

MEFE Site Visit 10.01.2015 MEFE Board Meeting Observations Guild_MEFE Contract 10_06_2014 FY16 MEFE Annual Site Visit

B.6 - To what degree does the authorizer have clear and comprehensive standards and processes to address complaints, intervention and corrective action?


Finding: The authorizer consistently implements clear and comprehensive standards and processes to address complaints and interventions; however, they do not have clear processes in place for corrective action. In the B.6 Narrative, the authorizer explains the process by which the Guild works to address complaints regarding the Guild and/or its authorized schools. This includes a notice of complaint by an identified complainant, a notice of receipt sent by the authorizer, a reasonable inquiry, and a response. The narrative states, “The length of the reasonable inquiry will vary depending upon the scope, complexity and specificity of the complaint.” A review of the authorizer’s website confirms

that complainants may submit a complaint by completing an electronic form. When asked during the interview, the authorizer reported that they have not had complaints filed with them or with MDE regarding their authorizing practices or any of the schools within their portfolio. Additionally, contracts with each school outline authorizer remediation and intervention actions. In all of the approved contracts with schools, the authorizer states that “if the school fails to achieve academic goals, financial performance, comply with Applicable Law, or other requirements, the Guild shall provide the following notices, as applicable.” The authorizer then outlines the different interventions, beginning with a notice to the school leader or board chair, which, if the situation is not corrected, is followed by a formal notice to the school board, and, finally, a notice to the school board of charter revocation/termination.

Although the AAA indicates that schools must submit a description of the school’s procedures for taking

corrective action in the event students achievement results fall below the goals approved by the authorizer in the charter contract, the authorizer does not describe its own corrective action process should a school fail to comply or meet the indicators outlined in the contract.

Key Evidence:

Minnesota Guild of Public Charter Schools Application (Approved Authorizer Application), pp. 27-30

Minnesota Guild of Public Charter Schools Operating Guide

Minnesota Guild of Public Charter Schools Complaint Form Guild Website Contact Page CSE July 2016 Board Agenda Complaint Submission Confirmation

06.24.2016 Guild_CSE.pdf CSE Union Guild press release 7-1-16 Edit#2


B.7 - To what degree does the authorizer support its portfolio of charter schools through intentional assistance and development offerings?


Finding: The authorizer does not consistently support its portfolio of charter schools through intentional assistance and development offerings. Although support and technical assistance is proactive, it is not provided consistently or in a variety of areas. In the B.7 Narrative, the authorizer stated that they host a training annually in conjunction with Epicenter representatives. However, the authorizer submitted documented evidence from only one Epicenter training that took place on July 20, 2016. This was verified by school directors during the interview. The narrative also explains that the authorizer has supported schools by “read[ing] critically and contribut[ing] to the grant narrative” related to Federal

CSP Grant Awards; however, the authorizer provided no documented evidence of this work. The narrative states, and participants reiterated during the authorizer interview, that the authorizer forwards emails from MDE to contacts at each school to ensure that they receive the required training. The authorizer also provided documentation of a training around special education funding. The authorizer did not submit evidence to verify to whom this documentation was sent and/or whether any of the schools in the authorizer’s portfolio attended.

Finally, according to school directors, the authorizer’s support and technical assistance is generally in response to problems and on an “as needed” basis.

Key Evidence:

B.7 Narrative Teacher Powered Conference 2017 SPED Training NACSA Principles_Standards EpiCenter Training 7.20.16 EpiCenter Overview School Director interview – October 12, 2016

B.8 - To what degree does the authorizer plan and promote the replication and dissemination of best practices of its high performing charter schools in its portfolio?


Finding: There is no intentional plan for successful model replication and dissemination of best practices. In the B.8 Narrative, the authorizer described hosting an evening on April 27, 2016 for schools during which they could come together to network before watching a movie (“Most Likely to

Succeed”) focused on a charter school. This was verified by school directors during the interview. One school director also stated that they have been linked to other schools who the Guild believes are excellent models. The authorizer provided one example of how it has connected two schools to share best practices. For example, it provided documents to show how it connected Lincoln International with Augsburg Fairview Academy as they were both preparing their board evaluations in order to collaborate. In the narrative, the authorizer also explains that it connected Rochester-Off Campus High School (ROC) with a high-performing, teacher-centered school in St. Paul; however, the authorizer did not provide any evidence that it had made this connection. Further, in post-interview documents, the authorizer indicated that four schools and two Guild advisors made a site visit to ROC on October 30,


2015, resulting in ROC teachers visiting Avalon and attending the teacher powered conference with teachers from the other schools; however, the document provided no clear evidence to show that the Guild initiated this collaboration between the four high schools. Although the authorizer may have created opportunities to promote the replication and dissemination of best practices, it does not have an intentional plan in place to ensure that these opportunities occur.

Key Evidence:

B8. Narrative Most Likely to Succeed Sign up EpiCenter Training Sign In EpiCenter Overview AFA_Lincoln Connection AFA Teacher Powered Screenshot AFA Introduction to Education Evolving 2015 Best Practice Award Application and Instructions School Director interview – October 12, 2016 GuildHSNetwork. pdf

PERFORMANCE MEASURES B: AUTHORIZER PROCESSES AND DECISION MAKING Authorizer Renewal and Decision Making

B.9 - To what degree does the authorizer have clear and comprehensive standards and processes to make high stakes renewal and termination decisions? To what degree do the authorizer’s renewal and termination decisions align to its stated renewal standards and processes and promote the growth of high-quality charter schools?


Finding: The authorizer does not have clear and comprehensive standards and processes to make high stakes renewal and termination decisions. Although the AAA and Operating Guide outline the process for renewal, they do not include transparent and rigorous standards and processes designed to use comprehensive academic, financial and student performance to make merit-based renewal decisions. The Operating Guide states that schools must submit a renewal application at least 180 days prior to the end of the current contract, schools must rely on the Guild Application for Contract Renewal, included in the Guide. The Operating Guide states that the authorizer will “conduct a formal evaluation

of a school during the final year of its contracts to prepare for the renewal application.” It explains that it

will “incorporate data gathered by the Guild, provided by the school and from external sources to

provide a holistic view of the school’s performance.” Additionally, the guide states that the authorizer will examine whether the school has met the terms of its contract requirements by meeting the performance indicators, improving student achievement, and achieving financial stability. Yet the authorizer does not provide comprehensive standards by which a school’s performance will be

measured. Further, other than the submission of an application and a decision by the board, the authorizer does not provide a clear process they will follow to evaluate schools to make renewal and termination decisions.


There is not sufficient evidence to evaluate the degree to which the authorizer’s renewal and

termination decisions align to its stated renewal standards and processes. Because the authorizer has not yet engaged in contract renewal with any of its schools (the first one will take place in FY 17), there is no evidence to show whether the authorizer’s decisions and resulting actions are consistent across

the portfolio or whether its decisions and resulting actions are aligned with its AAA. It should be noted that the authorizer terminated its contract with a school (TEAL) during the pre-opening phase only. According to the B.9 Narrative, the school was unsuccessful in securing funding (including the Federal CSP grant). Additionally, the school also had marginal enrollment, which led the authorizer initially to postpone the school’s opening date. After the Director (and founding member) of the school resigned,

the school board made a decision to dissolve the school. Therefore, sections of the formal closure plan were fulfilled, such as notifying parents.

Key Evidence:

B.9 Narrative Minnesota Guild of Public Charter Schools Application (Approved Authorizer Application), pp. 31-33

Minnesota Guild of Public Charter Schools Operating Guide, p. 38

Sample Contract FY16 ASA Annual Site Visit Exhibit F Charter School Closure Plan

Documents

Authorizing Plan (AAP) Guidance. - MDE...Minnesota Statutes, section 124E.05, Subdivision 5, “The commissioner shall review an authorizer's performance every five years in a manner