Department of Procurement Services

Date: November 28, 2017

Addendum II

RE: RFP#12152017 Decision Analytics & Information Management

1). The SCBE previously had an Ed-Fi based solution, what happened to it? Response: The Ed-Fi based solution proved to be unsustainable due to a number of issues including significant changes in source applications, reliance on substantial amounts of customization and lack of internal skillsets to maintain the application moving forward.

2). Was it limited in some way or cost prohibitive, which the SCBE is trying to rectify with this RFP? Response: Yes, and Yes. The Ed-Fi solution did not fully meet the expectation of district and school-level stakeholders. The requirements framed in the RFP are intended to address the gaps experienced with the Ed-Fi solution through a sustainable holistic architecture. This RFP is meant to solve underlying issues previously missed and provide a sustainable path forward for the district.

3). Page-18 (#11): Does the Decision Analytics Platform have to be built using Microsoft PowerBI or can it be used with the vendors’ BI tool? Response: The Decision Analytics platform does not have to be built in Microsoft PowerBI. The district recently transitioned to Office 365 and is looking to leverage that investment to the greatest extent possible. Our goal is to deliver a single point of entry for our end users, so they are no longer receiving pieces of data from multiple sources or having to figure out where they should go for different information points. Meeting

our end user needs (outlined on pp 5-7 of the RFP) in a sustainable and efficient manner is our focus with the RFP.

4). Page-18 (#12): Integration with Microsoft PowerBI (already licensed and installed) to provide Ad Hoc Visualization and Reporting Capabilities – Is this the only option for Ad hoc visualization and reporting, or can the vendor provide their own?

Response: The district expects to continue to use PowerBI going forward due to all SCS staff having access to leverage the tool through our Office 365 agreement. Alternative BI proposals should be explained and accounted for in the submitted proposal. The proposal will be assessed based on how well it meets the outlined requirements.

5). Specifically, since this is a non-construction contract, will a bond be required if the submitted bid is in excess of $100,000.00? Additionally, will a bond be required if the submitted bid is less than $100,000.00?

Response: We reserve the right to request bonding.

6). p. 8, Capabilities Overview & Architecture, Architecture Map - What existing Ed-Fi technologies (API, operational data store, XML ingestion) are installed at Shelby County Schools? Please specify the Ed-Fi versions and their level of deployment and use.

Response: The Ed-Fi ODS and dashboards are not currently active. Previous evaluation identified an upgrade to version 2.x would be required in conjunction with the remapping to multiple application changes, which are identified in the application listing in the appendix.

7). p. 8, Capabilities Overview & Architecture, Architecture Map - Are the Ed-Fi

dashboards installed at SCS? Have they been extended for risk indicators?

Response: The Ed-Fi dashboards are not currently linked to active data. A highly customized risk indicator was developed, but experienced limitations to only 9-12 and was not adaptive over time.

8). p. 8, Capabilities Overview & Architecture, Architecture Map - Are there any existing

connectors or integrations with the Ed-FI ODS, via the API or via batch XML?

Response: No APIs were used due to the ODS version in use 1.5. Usefulness will be limited due to the amount of application changes that have occurred. 9). p. 8, Capabilities Overview & Architecture, Architecture Map - What version of

PowerSchool will be integrated?

Response: PowerSchool version 11.x in the Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production environment.

10). p. 8, Capabilities Overview & Architecture, Architecture Map - Do you have an existing

data warehouse? What reporting is driven from the data warehouse? Is there a desire to source data into the data warehouse from the new Ed-Fi infrastructure?

Response: The inactive Ed-Fi ODS was our attempt at a data warehouse and is not use for any reporting. The warehouse solution should align with the proposed architecture.

11). p. 46, diagram provided in Appendix C - do all of the connections to/from PowerSchool

exist today? What is the makeup of each of those? API, batch, csv, etc.

Response: There are currently no APIs in use among any applications in our environment. IT does not have a comprehensive listing of connections between applications. Connections are functioned through a combination of batch, CSV and XML point-to-point data transfers that are stood up for each individual request. The proposed iPaaS should address the management of connections and data flows.

12). p. 21, 2.0 – Joint Ventures - Would SCS consider a “best-of-breed” solution that would

allow for the two main components of the solution, a.) Information Management, and b.) Decision Analytics, to be provided by separate, but complementary, vendors under multiple awards? Or is it important to SCS that all the components are provided in a single technical solution from one vendor or multiple vendors in partnership?

Response: SCS is looking for single technical solution. A best of breed, if proposed, should be submitted through a partnership.

13). p. 18, #15 – Tools for streamlined Federal and State Compliance Reporting - How is state and federal reporting accomplished today? Is there a database that collects data for that purpose? In what form is data sent to the state?

Response: Data is uploaded to the State EIS.

14). p. 16, #7 – Research-based and Validated Frameworks - Does SCS use any existing

research-based frameworks for at-risk indicators? If so, please provide details.

Response: The previous at-risk modeling was developed for SCS by an outside contractor. It focused on identifying students in grades 9-12 that were at-risk of not graduating on time. The model was dependent on the Ed-Fi ODS and dashboards that are no longer active.

15). p. 21, SECTION B – SCOPE OF SERVICES, Paragraph 1.0 The Services, Capabilities

requirements, Sub-Section F. Technical Requirements, Clause 20. Data Security

Policies, Procedures & Practices, bullet No. 4, states the following, “Ownership: Shelby County Schools maintains ownership of all data and associated business rules during the entire duration of platform usage, and maintains the right to have data and business rules returned or purged upon our request.”

Please confirm, is it the District’s intent that ‘ownership’ refers only to data and business

rules either owned by the District prior to the effective date of a final contract to be

awarded or to data and business rules created for the District during the performance of

the Services under the terms of such final contract? Also, to the extent such other data

and business rules are necessary for performance or provision of the Services, will the

district consider securing a license from the Respondent for use of all other data and

business rules owned by either the awarded Respondent or other third-parties?

Response: This statement refers only to data and business rules either owned by the

District prior to the effective date of a final contract to be awarded or to data and

business rules created for the District during the performance of the Services under the

terms of such final contract. We would consider securing a license if outside data is

required.

16). p. 21, SECTION B – SCOPE OF SERVICES, Paragraph 2.0 Joint Ventures, first sentence,

states the following, “Respondents are encouraged to enter into joint ventures for the

purposes of responding to this RFP and providing the Services.”

Please confirm, in addition to establishing Joint Ventures, will the District permit

Respondent to enter into one or more sub-contracting agreements with third-parties in

order to perform or provide the Services?

Response: SCS is looking for single technical solution. We encourage the respondent to

enter into Joint Ventures with Local Business, Minority Owned Businesses, Women

Owned Businesses

17). p. 22, SECTION B – SCOPE OF SERVICES, Paragraph 4.2 Description of the Respondent.,

Sub-paragraph 4.2.4, states the following, “Disclosure of the volume of sales the

Respondent has had in each of its past two fiscal years;”

If the Respondent is a non-public company, may the Respondent provide a sales range

as its response and can this specific response be claimed as an exemption from the

Tennessee Public Records Act?

Response: Yes, you can provide a sales range, however there will be no exemption.

18). p. 24, SECTION B – SCOPE OF SERVICES, Paragraph 5.0 Compensation., states the

following, “The successful respondent will be compensated upon the completion of the

project’s objectives and upon meeting or exceeding the District’s Scope of Services

requirements. To the extent the provisions of this paragraph contradict any provision

regarding compensation set forth in Section C of this RFP, the provisions set forth in this

paragraph shall govern.”

Will the District consider a compensation structure whereby the Respondent is

compensated based on mutually agreed to milestones and/or deliverables where such

milestones and/or deliverables are met or provided to the District prior to the

completion of all of the project’s objectives? Also, to the extent one or more licenses

are necessary to provide the Services, will the District compensate the Respondent for

use of such licenses prior to the completion of all of the project’s objectives?

Response: Yes, SCS will consider.

19). p. 24, SECTION B – SCOPE OF SERVICES, Paragraph 6.0 Bonding Requirements., states

the following, “The successful contractor shall be required under the Contract to comply

with the bonding requirements listed below, if any.”

As the contemplated Services do not include construction related services, please

provide details with respect to any specific bonding requirements associated with the

contemplated Contract.

Response: We reserve the right to request bonding.

20). p. 30, SECTION C – INSTRUCTIONS TO RESPONDENTS, Paragraph 9.3 Compensation.,

states the following, “The successful respondent will be compensated upon the

completion of the project’s objectives and have met or exceeding the District’s Scope

of Services required. More specific guidance regarding compensation may be set forth

in Section B of this RFP.”

Will the District consider a compensation structure whereby the Respondent is

compensated based on mutually agreed to milestones and/or deliverables where such

milestones and/or deliverables are met or provided to the District prior to the

completion of all of the project’s objectives? Also, to the extent one or more licenses

are necessary to provide the Services, will the District compensate the Respondent for

use of such licenses prior to the completion of all of the project’s objectives?

Response: Yes, SCS will consider.

21). p. 31, SECTION C – INSTRUCTIONS TO RESPONDENTS, Paragraph 9.6 Performance.,

states the following, “SCBE and the successful contractor each shall agree to fully

perform all of their respective obligations under the Agreement. However, failure of

SCBE to perform such obligations shall not automatically relieve the successful

contractor of its obligation to perform under the contract.”

If the provision or performance of certain deliverables by Respondent associated with

the Services are contingent upon the performance of certain obligations by SCBE (e.g.

payment obligations, review and approval by SCBE personnel of specifications or

deliverables, the provision of access to SCBE’s application environment and data, etc.),

please describe how SCBE would contemplate Respondent performing such contingent

obligations in a timely matter in the event SCBE fails to perform its obligations which are

necessary for Respondent to complete its obligations?

Response: Dependencies and required parallel milestones will be accounted for and identified through the project planning phase. The initiative will establish an on-going partnership with SCS and the selected vendor working collaboratively to ensure overall success.

22). p. 35, SECTION C – INSTRUCTIONS TO RESPONDENTS, Paragraph 10.22 Insurance.,

states the following, “By submitting a Proposal in response to this RFP, the Respondent

certifies that, if awarded a contract, it will have the insurance coverage required for

performance of the Services, if any, at the time the work commences.”

Please provide the specifications for the insurance coverage required, if any?

Response: Please reference Section 10.0 General Terms and Conditions, 10.22

Insurance.

23). p. 35, SECTION C – INSTRUCTIONS TO RESPONDENTS, Paragraph 10.24 Ownership of

Computer Programs and Data., states the following, “Ownership of all computer

systems, programs, software, data, materials, documentation or similar products

purchased, created or compiled in connection with the performance of the Services or

the performance of obligations under any contract resulting from or related to this RFP,

now or hereafter, shall vest completely and exclusively with SCBE. Upon expiration of

the term of the contract, the successful contractor will relinquish and convey to SCBE

any right it may have in such computer systems, programs, software, data, materials,

documentation or similar products.”

Certain intellectual property owned by Respondent or a third-party may be required to

provide the contemplated Services and fulfill Respondent’s obligations. Therefore, will

SCBE consider a term license to use certain computer systems, programs, software,

data, materials, documentation or similar products procured, created or compiled in

connection with Respondent’s performance of the Services or the performance of its

obligations under the contract contemplated to be awarded?

Response: We would consider securing a license if outside data is required.

24). Does real-time mean instantaneous or do you mean "near" real-time? What is

acceptable for real-time?

Response: “Real-time” is used in the context of actual stakeholder statements.

Currently, dashboards and most reports are only available with a minimum of a day lag

due to overnight load processes. Acceptable has not been clearly defined. A minimum

expectation is that as data entry/corrections are made in source systems there is an

established expectation for when it will be reflected in dashboards and reports within

the same day.

25). What tools and technologies do you currently use to track students? Who is responsible

for inputting data into these tools / technologies? How do the end-users and key

stakeholders currently use these existing technologies?

Response: The bulk of student information is entered and collected through our student

information system, PowerSchool. Individuals at the school-level are responsible for

data entry.

26). What data do you currently collect to support high school graduation and dropout

prevention?

Response: Tableau dashboards are in use to provide views of attendance, discipline,

grades and credit accumulation as individual dashboards. There is also Excel file based

tracking of student cohort lists to document students that have moved to other schools.

27). What data do you currently collect to support college and career readiness?

Response: ACT scores, TN Ready, NWEA MAP (as a 3x year assessment of progress),

Certification progress (for career readiness). All data is intended to align with the State

of TN standards for college and/or career readiness.

28). How do you currently identify at-risk students? At what age / grade?

Response: Our at-risk identification is not currently in use. When previously in use, it

covered 9-12 grade students. The intent is to be able to identify students at-risk of not

graduating on time and whether or not they are college or career-ready through

predictive characteristics, such as attendance, discipline, grades, assessment

performance and other measures.

29). How often do the structures of datasets change? What are some examples of this type

of change?

Response: Dataset structure changes might be from changes in the files the State sends

to SCS or source changes due to new application versions. A typical year might have 2-4

occurrences.

30). How often do your needs change? What are some examples of changing needs?

Response: Changes might be driven by changes in State requirements or as new

programs are stood up within SCS. New or expanded student programs are common

year to year within the district as new areas of focus are identified.

31). How do you define "like" organizations? Is your focus primarily on benchmarking against

k12 / educational organizations? Do you compare against industry organizations as

well?

Response: We expect to focus on primarily on other large, urban school districts.

32). What are all of the factors that will go into calculating percent up time (scheduled

maintenance, patching from vendor software, etc...)?

Response: Percent up time should be based on the percentage of time the solution is

live and accessible to users. We do not expect interruptions in availability due to

maintenance or version upgrades.

33). Can demonstrated experience in building accurate and impactful predictive analytics

models be used in lieu of existing historical models?

Response: Yes.

34). Which browsers and versions do you consider a modern browser?

Response: Current releases of Chrome, Firefox, Edge, Safari.

35). What disaster recovery and backup solutions do you currently use or plan to use?

Response: PowerSchool is operated through the vendor’s cloud environment and

follows their standard protocol. On-premise applications operate through a nightly roll-

over with a production and QA server refresh. As more of the application environment

moves to the cloud, it is intended to leverage the standards available through the cloud

environment of Oracle (future ERP) and Microsoft Azure.

36). What data retention requirements do you currently have?

Response: Retention requirements vary widely in areas of student and personnel

information. Not all data types have a defined and documented retention standard.

37). How is the SCS/board currently meeting this need? Which vendor(s) currently provide

the reporting and data warehousing services?

Response: There is some reporting integrated into existing source applications, but not a

single interaction point for all reporting. Tableau Server is in use for attendance,

discipline and grades dashboards available to district level users and school leaders. The

background information for the Tableau dashboards is manually pulled through a series

of stored procedures from the PowerSchool application. There is additional reporting

delivered through individual spreadsheets from a variety of sources.

38). Which analytical solutions are currently used? For this RFP to perform all required 5

types of analytics, any preference which statistical tools should be used such as SAS, R

etc.?

Response: Tableau and PowerBI are both currently in use. There is a small Research

group that leverages SAS/SPSS for statistical analysis.

39). Can some of the resources work remote on this project?

Response: Yes.

40). Any need to integrate Web APIs/Web services to extract or feed data from/to any 3rd

party system(s)? If yes, please provide details.

Response: There is a desire to include data from the National Clearing House and State

of TN systems. Currently, this data is received through flat file transfers.

41). Any preference on what cloud environment should be used for implementing the

solutions such as AWS, Azure, Google Cloud etc.?

Response: Our Office 365 is currently delivered through an Azure cloud environment.

42). What will be the project management process and engagement from SCS/board? Please

provide details.

Response: The project will be owned and run through the Strategy & Performance

Management division within SCS. We look forward to working with the selected vendor

during the contract negotiation phase to iron out detailed roles & responsibilities that

will ensure the overall success of the effort.

43). How large is the data size? How many years of historical data is available for predictive

analytics? Any preference on which modelling techniques should be used such as

Regression, Neural Networks, Decision Tree for predictive analytics?

Response: Looking at less than a terabyte of data. For historical data, it is recommended

only data after the merger & demerger years (2014-15 and forward) are used to provide

a representative student population.

44). Can SCBE identify the composition of the evaluation committee? How many

representatives will be on the committee and what are their roles within the school

district?

Response:

Title Department

Buyer Procurement

Financial Systems Advisor

Finance

Manager Information Technology

Manager Information Technology

Manager Performance Mgt.

Director Research & Performance Mgt.

Instructional Leadership Director

Office of Schools

Executive Director

Planning & Accountability

Manager Office of Schools

Instructional Leadership Director

Office of Schools

a. Can identify which department(s) will be leading this project?

Response: Performance Management (housed in Strategy & Performance Management)

45). The budget award for substantially similar projects for similar size school districts has

been approximately $850,000 in the first year and $100k - $250k annually for

subsequent years depending on the audience served, the amount of data to be

integrated and other factors. Is that budget comparable to SCBE’s budget for this

project?

Response: Yes.

46). Is this project funded through a grant?

Response: No.

47). Please clarify: how will the new Decision Analytics Information Management System,

including the dashboards, will work with the current Ed-Fi dashboards that SCBE has

already implemented, as described here: http://www.scsk12.org/edfi/? Will those

dashboards and data warehouse continued to be used and supplemented by this

system, or will they be sunsetted with the new system?

Response: Already sunsetted, see above questions on topic.

a. If the Ed-Fi data warehouse will still be used, where in the architecture diagram

on page 8 of the RFP does that data warehouse fit in?

Response: n/a

48). At the recent Ed-Fi Summit, Chris Moffatt, Ed-Fi’s Director of Technology pointed to

some challenges with the Ed-Fi Operational Data Store, including that a separate

Operational Data Store must be implemented for each school year. As a result, any year

on year or other longitudinal query would have to navigate the complexities of that

architecture. Is longitudinal data a priority for SCBE? Is that what Shelby County wants

or is it seeking a high performing, proven, sustainable education data warehouse from

an established ed tech vendor?

Response: Longitudinal query ability is a REQUIREMENT. Any solution unable to deliver a

longitudinal view of an individual student or cohort of students would not meet the

minimum requirement of the RFP.

49). Please confirm: Per the diagram on page 8 of the RFP, SCBE is seeking a Microsoft SQL

Server Data Warehouse and not an Ed-Fi ODS as part of the proposed solution.

a. If SCBE is seeking an Ed-Fi data warehouse, we are having a hard time identifying

a fully implemented Ed-Fi data warehouse, even though Ed-Fi has been around

for over 6 years. If SCBE wishes to have an Ed-Fi data warehouse as part of this

solution, can the district point us to one based on Ed-Fi as an example? We want

to model our proposal off a proven successful data warehouse implementation.

Response: We are interested in successfully following the Ed-Fi data standard. Our

preferred environment is Microsoft SQL due to current district skillsets and environment

uses of related products.

b. What is SCBE planning on doing with its existing Ed-Fi data warehouse?

Response: The existing Ed-Fi ODS is not operational due to multiple changes in source

applications and a version upgrade requirement to the 2.x releases.

50). Appendix C: Can we assume that all of the district’s data sources listed in Appendix C

will provide data in the Ed-Fi data standard? If so, which version of the standard will

they adhere to?

Response: The expectation is that part of the proposed solution will account for

transforming source data to the Ed-Fi data standard. All data sources will not necessarily

be covered by the Ed-Fi data standard. For data not covered by the Ed-Fi data standard,

the source systems standard and definitions should be carried through to the data store.

The current Ed-Fi data standard is version 2.1.

51). Regarding the Ed-Fi requirements, would the district consider a proven education data

warehouse solution used by districts serving over 20 million students throughout the

United States that aligns with the Ed-Fi data elements, but does not use the Ed-Fi

standard?

Response: What is the differentiation between Ed-Fi data elements and Ed-Fi data

standard? The Ed-Fi platform is NOT required to deliver the Ed-Fi data elements and

standard.

52). Page 8: Please clarify: how does SCBE currently use the Impact TN Ed-Fi Dashboard I

identified in the diagram on page 8?

Response: SCS is not currently using the Impact TN Ed-Fi Dashboards. The intended path

forward from the TN DOE is to sunset EIS and move all school districts to an Ed-Fi ODS.

The state reporting part of the any proposal should account for how the State of TN DOE

will require data be submitted to them.

a. Besides the Impact TN dashboards, are there other Ed-Fi standards-based

solutions that the system will feed data into?

Response: None currently planned or in-use.

53). Regarding PowerBI (several places in the RFP): Does SCBE anticipate that the out-of-the-

box dashboards use the PowerBI platform, or will PowerBI only be used for the

advanced analytics and ad-hoc reporting?

Response: Either approach is acceptable. A key interest is in providing as seamless an

interaction as possible for our end users at the school-level, while delivering the

actionable insights required. How a proposal most effectively achieves the requirements

outlined in the RFP is up to the individual vendor.

54). Regarding data access and management on page 15: How many users does SCBE

anticipate having for each type of defined role?

Response: The largest defined role type is that of our teachers (approximately 6,500

district-wide). Additional role types, such as Counselors, Assistant/Vice Principals,

Principals and District personal total less than 2,000.

55). Regarding the timeline described on page 13, with the anticipated implementation date

of July 15, 2018, what are the district’s priorities and preferences on how it would like to

roll out the new system to users?

Response: At-Risk Student identification is the highest priority. Project success is having

training and adoption processes in place prior to the 2018-19 school year. A school-level

adoption plan is part of this priority. Our primary goal is to deliver actionable

information in an easy to digest form to our teachers and school administrators from

the start of the year. SCS is also currently implementing the Oracle Fusion ERP and must

not be interrupted.

56). General solution question: In addition to a production environment, does SCBE also

want a training/test environment?

Response: Yes.

57). General solution question: Is SCBE expecting to migrate longitudinal data into the new

system? If yes, how many years and for which sources?

Response: Yes, back to the 2014-15 school year. We want to make this as complete a

picture as possible, but the capability to bring this information from all sources will have

to be evaluated as part of the initial assessment.

58). Regarding the Training and Support Plan described on page 19: What District resources

would be involved in the project and to what extent (i.e. how much of an FTE for each

role)?

Response: The Performance Management team (5 FTE) will own and manage the

project. There will be partial FTE from other teams to support adoption and

communication across the district.

59). Regarding the Training and Support Plan described on page 19: Does the district have a

Level 1 Help Desk? Is the vendor’s support line expected to be Level 1 or Level 2

support?

Response: Yes, there is a Level 1 Help Desk that covers general access issues. Vendor

support line should cover troubleshooting and usage questions specific to the delivered

solution. A robust FAQ and training plan is part of a successful support plan to minimize

end user tickets.