1. Introduction - vpfa.boisestate.eduvpfa.boisestate.edu/process/review/biddocs/KK19... · Web viewAll submitted material becomes the property of the University and will not be

Request for Information (RFI) KK19-146 for Course Evaluation System (CES)

Table of Contents1. Introduction 2

1.1 Overview of Boise State University 2

1.2 Project Overview 3

1.3 Key Steps and Timeline 4

2. Instructions to Vendors 4

2.1 Response Requirements 4

2.2 Vendor Questions 5

2.3 University Contact 5

2.4 Public Records 5

3. Requirements 5

3.1 Vendor and General Product Information 6

3.2 Functional Requirements 6

3.2.1 Question Development 6

3.2.2 Evaluation Administration 7

3.2.3 Evaluation Participation 8

3.2.4 Reporting 9

3.2.5 Managing Evaluation Data 11

3.3 Technical and Support Requirements 13

3.3.1 Technical Requirements 13

3.3.2 Support Requirements 15

3.4 Cost Information 16

3.5 Implementation Information 17

4. Additional Information and Future Capabilities 17

1 | Page

1. IntroductionBoise State University (“University”) is seeking information on Vendor capabilities and comparative costs for a campus-wide solution course evaluation (CES) system (“System”). This document is intended to enable Vendors to provide information regarding their CES. This is an informal action initiated at the request of Boise State University.

The information obtained from this RFI may be used in developing further specifications and could result in a solicitation for the implementation of a course evaluation system. All parties responding to this RFI will be notified if there is solicitation issued. Vendors may check the University Department of Purchasing Bid Opportunities webpage at https://vpfa.boisestate.edu/purchasing/ for current bid opportunities and updates on this RFI.

1.1 Overview of Boise State UniversityBoise State University is a comprehensive public university offering about 200 programs of study, including 43 fully online programs.

2018 Figures for Boise State University

25,540

16,995

Total Students

-- Full Time Equivalent

19,361

15,883

2,824

654

Total Degree-Seeking Students

-- Undergraduate

-- Graduate

-- Second Undergraduate Degree

10,890

4,650

1,471

4,769

Total Classes Offered for 2018 Calendar Year

-- Spring 2018

-- Summer 2018

-- Fall 2018

181,978

79,683

16,545

85,750

Total Student-Class Enrollments for 2018 Calendar Year

-- Spring 2018

-- Summer 2018

-- Fall 2018

1,526

1,013

Total Faculty

-- Full Time Equivalent

In the same academic year, there were 757 full and 769 part time faculty members.

2 | Page

An academic year includes three semesters--Spring, Summer, Fall. In each semester there are multiple academic sessions of different durations in which classes are held that result in approximately six evaluation periods in the Spring and Fall, and some four evaluation periods in the Summer. In addition, there are numerous “Special Session” credit-bearing workshops that occur at random times and durations. These are evaluated in ten to twelve further evaluation periods in which workshop end points are clustered. In Fall 2018, a total of 18 evaluation periods were included.

Evaluation questions are determined for the most part at the college and department levels. A subset of six questions are used by a majority of classes, but there are no truly universal questions. Some blocks of questions are determined by academic support units, such as Honors program, Extended Studies-online courses, Service Learning, etc., and are attached to classes that share certain attributes. In some cases, questions are determined by course coordinators for specific courses.

The operational management of all questions, data, set up, etc. is accomplished by application administrators in the Institutional Research office and in Extended Studies (for Special Sessions). In all, about .5 FTE are required to administer the current system--two administrators dedicating 20% and 10% of their time, and a manager dedicating 20% of his time.

Boise State University integrated enterprise systems include the following:

o Student Information System (SIS) – PeopleSoft Campus Solutions 9.2o Learning Management System (LMS) – Blackboard Learn Course Delivery (3500.7.0-

rel.10+58ec50a)o Business Intelligence Systems

▪ SQL Server 2016

▪ Oracle Automated Analytics Cloud 12.2.5.0.0 (Build BIPS-20180909211754 64-

bit)

▪ Blackboard Analytics for Learn (A4L) version 4.3.3

o Content Management System – Perceptive Content Client version 7.2.2.3338

1.2 Project OverviewBoise State University seeks information about the acquisition of a new CES system. University has been conducting online course evaluations on over 95% of its credit-bearing courses for the past nine years. Long term goals include a system that maximizes information for use by instructors to improve their teaching and by administrators to evaluate instructor performance while reducing the cost and administrative effort to operate.

The CES system will serve our entire University for evaluation of credit-bearing courses.

Project objectives are as follows:

● Make available to departments and colleges a robust, convenient, comprehensive, and mobile-

friendly course evaluation system.

3 | Page

● Ensure that the chosen System includes all required characteristics and as many desirable,

optional characteristics as possible.

● Accomplish the set-up, loading, and operation of routine evaluations with minimal resources.

● Make easy-to-use reporting and analytics tools available to faculty for gaining deeper insights

into the strengths and weaknesses of their instruction.

1.3 Key Steps and Timeline Timeline

Deadline to receive questions: March 29, 2019

Anticipated release of answers to questions: April 3, 2019

RFI Closing date: April 16, 2019, 5 pm, via email.

● The Boise State University Department of Purchasing will post this RFI on or about 3/22/19.

● Vendors must email questions to the University Contact per the instructions listed in Section 2.

● Based on RFI responses, a short list of Vendors who appear to best match the needs of the

University may be asked to present demonstrations of their products. University expects demos be face to face, via webinar, or a blend of face to face and web-based. Demos will be at the sole cost of Vendor.

● University will review responses and demos and decide whether to pursue a solicitation.

● Information pertaining to this RFI is available at the University Department of Purchasing Bid

Opportunities webpage at https://vpfa.boisestate.edu/purchasing/

2. Instructions to Vendors2.1 Response Requirements

Please provide electronic responses to questions and requirements in the Requirements Section (Section 3) of this document in accordance with the dates listed in the Key Steps and Timeline (Section 1.3). Also attach any brochures or other written information pertinent to the University requirements.

Vendor must respond to each question in the RFI using the response codes below. In addition, several questions ask you to describe Vendor-offered functions. If a question includes “PLEASE DESCRIBE” or “LIST” in bold type, it requires additional information. Unanswered questions will carry the assumption that Vendor does not in any way provide the functionality.

Response Codes:

4 | Page

Y - Yes, feature is currently supported by the softwareN - No, feature is not currently supported by the softwareC - Custom development for implementation would be required. Please note if the cost is included in stated System price (C-Included) or would require an additional purchase (C-Added).F - Future baseline feature planned by Vendor. Note: Provide anticipated release date.

Vendor may delete the first two sections of this RFI, and simply return Sections 3 & 4 as the RFI Response.

2.2 Vendor Questions If Vendors have questions on anything contained within the solicitation, they may email the University Contact. The email must contain the subject line “CES RFI Questions - from (Vendor Name)” and must be received by the time and date listed in the Key Steps and Timeline (Section 1.3).

University will post answers to Vendor questions in an amendment to this RFI, at the University Department of Purchasing Bid Opportunities webpage at https://vpfa.boisestate.edu/purchasing/

2.3 University ContactThe RESPONSE MUST BE EMAILED to [email protected] and must be received by the date listed in Key Steps and Timeline in Section 1.3.

If you have any questions, contact Kristen King at [email protected] or 208-426-1802 at the Boise State University Department of Purchasing.

All documentation regarding this RFI will be posted at the Boise State University Department of Purchasing Bid Opportunities webpage at https://vpfa.boisestate.edu/purchasing/

2.4 Public RecordsAll submitted material becomes the property of the University and will not be returned to Vendor. Submitted material and/or supporting documentation may be available for public inspection upon written request, except for information specifically labeled on each separate page as a “Trade Secret” under the Idaho Public Records Act, Section Title 74, Chapter 1, Idaho Code (“the Act”). Alternatively, information may be specifically labeled “exempt” from public records under another exemption found in the Act. Information specifically labeled as Trade Secret or otherwise exempt may be protected from disclosure, but only to the extent consistent with the Act or otherwise applicable federal or state law or regulation. Accordingly, the University cannot guarantee its confidentiality.

3. RequirementsPlease supply information about the following categories, reserving discussion about future capability for the Additional Information Section (Section 4):

● Vendor and General Product Information (Section 3.1)

● Functional Requirements (Section 3.2)

● Technical and Support Requirements (Section 3.3)

5 | Page

● Cost Information (Section 3.4)

● Implementation (Section 3.5)

3.1 Vendor and General Product Information

Question Response

Vendor Information

Company name

Parent company

Company address

Name of person responsible for the information contained in this RFI

Telephone number

Fax number

Email address

Web page

General Product

Name and version of CES recommended for Boise State University

Year product(s) introduced on market

Total number of installations of the version of the software being proposed.

Approximate market share

Universities currently using your system for CES

3.2 Functional Requirements

Please use the response codes below to address each functional requirement question. If a question includes PLEASE DESCRIBE or LIST in bold type, it requires additional information. Unanswered questions will carry the assumption that Vendor does not in any way provide the functionality.

Response Codes:Y - Yes, feature is currently supported by the software

6 | Page

N - No, feature is not currently supported by the softwareC - Custom development for implementation would be required. Please note if the cost is included in stated System price (C-Included) or would require an additional purchase (C-Added).F - Future baseline feature planned by Vendor. Note: Provide anticipated release date.

3.2.1 Question Development

Requirement Response

Required

DYNAMIC EVALUATION -- Can the evaluation for a class be dynamically built from the various blocks of questions that match various characteristics of the class? For example, a honors math class taught online might have sets of questions drawn from the Math department, the Honors College, and Extended Studies (which manages online courses).

INSTRUCTOR ADDED QUESTIONS --Can an instructor easily add their own questions to the standard questions their class regularly includes?

QUESTION TYPES -- Are various question types available, including at least Likert scale, open ended, multiple choice (single and multiple answer)? PLEASE LIST QUESTION TYPES.

CONDITIONAL QUESTIONS -- Is it possible to build conditional questions which are displayed only upon certain responses to prior questions?

Optional

QUESTION BANK -- Is a Question Bank available from which to select questions? The "Bank" could be made up of high quality questions from other schools, or questions that we have pre-loaded for instructors to select.

OTHER SCENARIOS -- Are there other question-related scenarios that would highlight the advantages of your system? If so, PLEASE INSERT.

7 | Page

3.2.2 Evaluation Administration

Please use the response codes below to address each question. If a question includes PLEASE DESCRIBE or LIST in bold type, it requires additional information. Unanswered questions will carry the assumption that Vendor does not in any way provide the functionality.

Response Codes:Y - Yes, feature is currently supported by the softwareN - No, feature is not currently supported by the softwareC - Custom development for implementation would be required. Please note if the cost is included in stated System price (C-Included) or would require an additional purchase (C-Added).F - Future baseline feature planned by Vendor. Note: Provide anticipated release date.


Required

ACADEMIC SESSIONS -- Can the system support multiple, overlapping academic sessions with different date ranges for students to take evaluations? (E.g. There may be a 5 week, 8 week, and 16 week sessions in progress concurrently during the Fall Term.)

SCHEDULING -- Can the system allow scheduling academic sessions that include date ranges for students to take evaluations, and separate dates for allowing faculty access to reports?

RELATIVE SCHEDULING -- Can the system allow scheduling notifications relative to the Term or Session schedule dates? (E.g. 7 days before session start date, or 4 days after session end date.) This will allow us to avoid having to re-schedule every notification for every session and term.

NOTIFICATIONS -- Is the system able to send email notifications to students, requesting that they complete the evaluation, and to instructors, asking them to enter questions or other aspects of the evaluation?

REMINDERS -- Can the system send multiple reminders to students who have not yet completed

8 | Page

an evaluation?

CROSS-LISTED COURSES -- Will the system consolidate the results from 2 or more cross–listed courses into a single evaluation? (E.g., a single course with an undergrad and a grad designation)

CO-INSTRUCTORS -- When a course is team-taught, will the system enable the student to answer course-related questions one time, while answering instructor-related questions once per instructor?

Optional

MULTIPLE EVALUATIONS PER COURSE -- Can the system be used to evaluate a given course multiple times (e.g., a mid-course and a final evaluation) during an academic session. If Yes, can it then display a comparison of the results?

MISSING QUESTIONS -- Can the system check for scheduled classes that are missing evaluation questions and notify the administrator?

3.2.3 Evaluation Participation




Required

PARTIAL COMPLETION -- Is the student able to partially complete an evaluation during one sitting,

9 | Page

then return at a later time to finish it?

CLASS LISTING -- Can students access their evaluations for all courses from a single site that lists their classes, instructors, and other pertinent information?

SINGLE COMPLETION -- Is it possible to prevent students from completing a given evaluation more than once?

ANONYMITY.1 -- Can the system guarantee anonymity to students who fill out the course evaluations?

ANONYMITY.2 -- Can the system guarantee that there is no way that anyone at University could identify the student who authored a given ballot?

IDENTIFY STUDENTS -- Can Vendor, in the rare occasion that threats are made via the evaluation entries, identify the student who made the threat and relay that information to Boise State?

Optional

INSTRUCTOR POOL -- In a few cases students are not assigned to an instructor until after registration. Is it possible to enable students to select their instructor from a pool of instructors about whom they will fill out the evaluation?

CLEAR EVALUATION -- Prior to the end of an evaluation period, can the System enable a student to cancel/clear their submitted evaluation and to fill out the evaluation again?

3.2.4 Reporting


Response Codes:Y - Yes, feature is currently supported by the software

10 | Page

N - No, feature is not currently supported by the softwareC - Custom development for implementation would be required. Please note if the cost is included in stated System price (C-Included) or would require an additional purchase (C-Added).F - Future baseline feature planned by Vendor. Note: Provide anticipated release date.


Required

REPORT TEMPLATES -- Will the system allow an administrator to create/customize reusable standard report templates?

HIERARCHICAL REPORT ACCESS -- Will the system allow defining access to results based on a class ownership and/or personnel reporting hierarchy (E.g. chair is assigned access to classes owned by their department and/or to classes taught by faculty that report to him/her) as well as permitting more customizable, fine grained, rule-based access? PLEASE DESCRIBE the general options for how roles and permissions for access to results are defined.

COMPARE AGGREGATE -- Can reports compare class scores to aggregate scores for a department, college or institutional mean scores? PLEASE IDENTIFY any other types of aggregate comparisons that might be included?

LONGITUDINAL REPORTS -- Will the system allow an instructor to view longitudinal results for a course that was taught across a specified number of terms?

SUMMARY REPORTS -- Will the system allow creating a summary report that combines results across a set of classes? (E.g. Can you combine results for all classes in a department, all sections of a course, all upper division courses, all courses taught by a particular instructor?)

DELAYED REPORT ACCESS -- Can the system delay instructor access to reports until a time set after the evaluation period is closed?

NOTIFICATION -- Can system automatically notify

11 | Page

instructors when reports are available?

MINIMUM CLASS SIZE -- To protect anonymity, can the system automatically restrict the display of results to instructors if the number of student respondents does not meet a minimum threshold?

REPORTING AND ANALYTICS: PLEASE DESCRIBE how robust this feature is and how it will enable the University to develop insights based on aggregated, trended, and/or comparative evaluation results.

Optional

RAW DATA EXPORT -- Can the system enable an instructor to download their own course raw data (respondents = rows, questions = columns) in common formats?

CUSTOMIZE REPORTS --Can the system allow an instructor to customize the look and feel of their own reports?

REPORT GRAPHICS -- Does the system include both tabular and graphical representations of quantitative results?

AD HOC ANALYSIS -- Does the system have built-in tools for ad hoc quantitative analysis? If yes, PLEASE DESCRIBE.

QUALITATIVE ANALYSIS -- Does the system have built-in tools with which to analyze qualitative data? (E.g., word clouds, tools to categorize comments, add annotations, key word search, sentiment analysis). If yes, PLEASE DESCRIBE.

CROSS INSTITUTION -- Does the system or Vendor make available comparative data from other institutions? If yes, PLEASE DESCRIBE.

FILTER RESPONSES -- Can the system use closed-ended question responses to filter corresponding open-ended questions responses? (E.g., show all comments for those who rated the instructor as unsatisfactory).

12 | Page

USER ACCESS REPORT -- Does the system have a report to conveniently document access capabilities conferred within the system.

3.2.5 Managing Evaluation Data




Required

SYSTEM INTEGRATION -- Will the system interface with our PeopleSoft 9 implementation through Oracle Integration Broker or through an API or Web Services? The purpose of this is to extract enrollment data including students, classes, instructors, contact information, class characteristics, etc. PLEASE DESCRIBE preferred practices in integrating your system with PeopleSoft Campus Solutions.

FILTER OUT CLASSES -- Will the system allow filtering courses to exclude classes with certain characteristics from participating in an evaluation?

REDACT COMMENTS -- Will the system allow the instructor to flag entries, and for an administrator to redact inappropriate comments from appearing in instructor reports?

13 | Page

AUXILIARY DATA -- Will the system import, then export with results, additional data describing students, instructors, and/or courses? (E.g., student age or gender, course type or level, instructor type or gender). This additional data is used for post-hoc analytical purposes.

Optional

FILE LOADING -- Will the system allow for extracting enrollment data directly from simple spreadsheet files or databases in special circumstances?

DATA WAREHOUSE INTEGRATION -- Will the system integrate with our Business Intelligence Reporting data warehouse and tools?

HISTORICAL DATA -- Can historical evaluation data (obtained via a prior system) be imported and accessed from a centralized location with current/new course evaluations?

3.3 Technical and Support Requirements

3.3.1 Technical Requirements




Required

OFFSITE HOSTING -- Is the System a SaaS application delivered over the Internet, where

14 | Page

users can access it from any Internet-enabled device and location?

STORAGE-- PLEASE DESCRIBE your storage infrastructure, and any costs and policies associated with data storage and transfer.

WEB-BASED -- Is management of the system fully web-based so it does not require additional client software or the installation of browser plug-ins?

NO BROWSER OR PLATFORM RESTRICTIONS -- Is the System used by students, faculty, staff and employees browser and platform independent?

MOBILE --Is the System mobile-friendly for student and faculty user?. (The primary application administration interface does not need to be mobile-friendly.)

DATA RIGHTS -- Will the University retain the right to directly access the data and database or be able to extract all data?

DATA EDITING -- PLEASE DESCRIBE how data retention and deletion is managed by the system so Boise State University policies and business practices can be followed.

DATA OUTPUT FORMAT -- Is data export accessible via XML, CSV, Excel, and/or PDF or another industry standard?

SECURE TRANSMISSION -- Is the System and data integration secured through TLS, certificates, and other industry standard security practices? PLEASE LIST all that the System supports.

AUTHENTICATION INTEGRATION -- Does the system authentication use an SSO protocol via SAML 2.0/OpenID Connect?

ACCESSIBILITY -- Does the System meet WCAG 2.0 accessibility recommendations?

DATA PROTECTION -- Does Vendor provide documentation of physical and logical protection

15 | Page

of Boise State’s data and allow for auditing?

DATA BACKUPS -- Is data backed up on a regular and documented basis?

POLICY COMPLIANCE -- PLEASE DESCRIBE how the System adheres to all relevant legal requirements, such as the Family Educational Rights and Privacy Act (FERPA) and the Health Insurance Portability and Accountability Act (HIPAA), to ensure personally identifiable student information is not provided to unauthorized entities.

INTEGRATION: PLEASE DESCRIBE AND LIST your integration practices. Boise State University prefers that industry standards are used.

USER FEATURE ACCESS: Will system allow assigning access permissions for features and capabilities based on user roles or to individual users?

DKIM: Does system utilize DKIM (Domain Keys Identified Mail) or similar configuration to authenticate email sent from the System, to make email servers recognize that an email is from the University and not from a spam account?

Optional

SERVICE LEVEL AGREEMENT DETAILS -- A service level agreement covering uptime, availability, and capacity should be available and enforced. PLEASE DESCRIBE your standard service level agreement and data to show to what extent you have met it in the last six months. Are there any reports or reviews that you do with clients to ensure SLA’s are being met? Are there any consequences (e.g., prorating costs) if SLA is not met?

16 | Page

3.3.2 Support Requirements




Required

END USER -- PLEASE DESCRIBE the principle means, inclusive of the System cost, for End User training and addressing questions/issues (i.e. Student Users, Faculty Users, Academic Administration Users.)

APPLICATION ADMINISTRATOR -- PLEASE DESCRIBE the principle means, inclusive of the System cost, for university application administrators to obtain training, issue resolution and escalation, and technical support.

Optional

PILOT -- Is it possible for the client to conduct a real-use pilot test implementation of the system prior to purchase?

INSTITUTION INSTRUCTIONS --Can the system allow the institution to provide access to customized instructions specific to their institution from within the System?

If yes, PLEASE DESCRIBE .

TECHNICAL SUPPORT -- PLEASE PROVIDE LINKS AND/OR DESCRIPTIONS of available technical and customer support resources. E.g., Is free customer (tier 1) support included in your standard license agreement? What is the customer (tier1) support desk's

17 | Page

days / hours of operation? Is support available to specific contacts at the client institution or any/all end users?

ADVANCED TECHNICAL SUPPORT -- PLEASE ANSWER: Is advanced technical support (tier 2) included in your standard license agreement? What is the technical (tier 2) support desk's days / hours of operation?

INTERRUPTION OF SERVICE -- PLEASE ANSWER: In the event of an interruption of your service, what is your process for notifying customer operations of the circumstances of the interruption or outage and the expected recovery time?

SYSTEM ISSUES & OUTAGES -- PLEASE LINK to a transparent, public site where you publish any system issues or outages for everyone to see (if available).

DISASTER RECOVERY -- PLEASE DESCRIBE your disaster recovery strategy. How frequently is it tested?

3.4 Cost Information

For University planning and budgeting purposes, please provide a rough, non-binding estimate of costs for a university the size of Boise State (see section 1.1). Please answer each question in the space after each question.

● Please BREAK DOWN and DESCRIBE all costs that would be incurred by Boise State:

● Clearly DIFFERENTIATE the startup costs from annual or ongoing costs, maintenance costs and required from optional costs. Include items such as training and support in addition to software licensing:

● Please IDENTIFY any types of discounts offered:

● DESCRIBE how your product is licensed (Campus license, per user license, tiered pricing, etc.):

We are interested in assessing the total cost of ownership for your product. We would appreciate any information you can provide to help CLARIFY the divisions and tradeoffs between purchased products or services and University investments in staff or systems:

18 | Page

3.5 Implementation Information

Please answer each question in the space after each question.

● DESCRIBE your overall implementation philosophy and key considerations for implementation

planning:

● DESCRIBE your implementation approach. (Or include file name if discussed in an attachment.)

Include key activities, specific deliverables and milestones, timeline, both for the first-phase implementation and University-wide roll out, and resource requirements. For the timeline, include an estimate of the number of weeks the implementation team will take to complete a project of this scale. Include key assumptions and resources the University should be aware of when reviewing the overall implementation approach and timeline:

● PROVIDE the composition of the team for the implementation project:

o Key Vendor roles and the responsibilities fulfilled.o Key University roles recommended, including skills and experience required for

anticipated team members.o Estimated Time commitment (in hours) that University will need to dedicate to the

project.

● DESCRIBE any plans to “partner” with another Vendor to meet implementation needs. If your

approach includes the use of one or more additional Vendors or sub-contractors, please provide a detailed explanation of their role on the project. In addition, if your response to the technical and functional requirements and associated product demonstration is dependent upon a product offered by another Vendor partner, please be advised that a single, joint response should be submitted. Additional Vendors, subcontractors and/or any assignee or transferee must be able to adhere to the same agreements required of your company.

4. Additional Information and Future CapabilitiesPlease provide any additional information that you believe is relevant to helping us understand your solution, including future directions and special capabilities:

19 | Page

Documents

1. Introduction - vpfa.boisestate.eduvpfa.boisestate.edu/process/review/biddocs/KK19... · Web viewAll submitted material becomes the property of the University and will not be