September 30, 2014

Dawn Cullity

Bernadette Ferro

Kent Phillippe

VFA Year Two: Show & Tell

Then

VFA Overview

Now

The Voluntary Framework of Accountability is the first national system of accountability

SPECIFICALLY

for community collegesby community colleges.

User Experience

AACC’s Objectives for the VFA Improve the user experience

Increase utility (so we can meet all the goals listed here)

• Value-added metrics for accountability (using data to measure performance and make better decisions)

• National benchmarking for good practices

• Opportunity to streamline reporting / relieve IR capacity

• Ability to tell a better, more robust story

• Better data to inform public policy and advocacy

Utility

feedback

Objective: Improve Data Input

Feedback on the user set-up and data collection process/data tool

Stability of, and access to, the online collection and display (dashboards)

Tool is clunky, especially the online forms, and needs better validation processes

Workflow is not intuitive and progress bars does not tell us where we are in data collection process

improvements

Cohort Bulk Upload Online Data Entry

Raw Data File Upload

Not Reporting

2008 Cohort (6 year) O O O O

2012 Cohort (2 year) O O O O

CTE O O Not Available O

ABE O O Not Available O

Submit

Workflow: Data Configuration

Workflow: Managing Data CollectionWORKFLOW ACTIONS STATUS / NOTES

Configure Data “Data last configured on…”– with a list data configured (6 year, 2 year, CTE, and/or ABE) or

“Data needs to be configured”

Upload Data File(s) If no files to be uploaded show nothing.

If files have been uploaded “data last uploaded on .../../..” and lists each file type that has been configured and the date uploaded

Edit/View Data Dimmed if data not configured – “Data needs to be configured”Dimmed if data upload files are in the queue – “Data in the queue to be uploaded”If all data is Processed – allow user to select.

Data Validation Checks

Mark Data for Review If Data configured – allow for the user to click Mark for Review – ‘Display the last notes stored by the users of the system for review’

Publish Data

Dashboard and Exports Takes user to Exports page

Public Outcomes Reports Dimmed if data has not been made public yet

VFA Manual Data Entry

VFA Cohorts Disaggregated

feedback

Objective: Increase Utility of VFA Exports

Feedback on the use of the VFA and its data

Dashboards need improvement to be useful and understandable

Colleges need to be able to download final values including data and calculated percentages

Benchmarking parameters and policies

Dashboards need a lot of interpreting before passing to senior leadership/board improvements

VFA Export: Outcomes Dashboard

VFA Export: Tabular Data

VFA Benchmarking Dashboard: Benchmarking Selectors

VFA Benchmarking Dashboard: Six-Year Outcomes

VFA Export: Public Dashboard Report

feedback

Objective: Improve Data Collection Resources

Feedback on the resources (instructions)

Clarity: make the Metrics Manual user friendly, understandable

Consistency

Authority: be definitive with methodology and how metrics are to be reported

improvements

Cohorts

Reporting VFA

Measures

Six Year Cohort

Two Year Cohort

Main

Credential Seeking

First Time In College

Developmental Education Progress

Measures

Two-Year Progress Measures

Six-Year Outcomes Measures

Career & Technical Education Measures

Adult Basic Education Measures

CTE Cohort

ABE Cohort

VFA Metrics Manual

VFA Metrics Manual (cont.)

VFA Overview & Training Webinars

http://vfa.aacc.nche.edu/webinars/Pages/default.aspx

Y2 , Y3, Y4…

Close-Out What do you think should be our priorities going forward?

Questions & Feedback

Contact VFA at VFA@aacc.nche.edu

principle accountability

system

Reminders Sign-ups open October 1st

Data collection opens November 3rd

Questions During the Webinar

Data Dashboards Have the dashboards been tested with IE, Firefox, and Chrome?

The VFA dashboards are supported on the following Internet browsers: Internet Explorer (IE 8 – IE 10) Firefox (16.x – 22.x) Google Chrome (22.x – 28.x)

Please clarify public vs private dashboards. Upon publishing your VFA data, the VFA makes a link available on your College Profile page that takes the public user

to your Outcomes data. The Outcomes data are presented to the user in a packaged PDF report, again made available on a link from your College Profile page. All information that is contained in your Outcomes Dashboard is packaged into this public PDF report. The Outcomes data includes the “All Students” (not disaggregated) counts of the following: Developmental Education Progress, Two-Year Progress, and Six-Year Outcomes Measures; CTE Profile & Completer/Leaver Outcomes, and ABE Cohort Outcomes. Raw data, disaggregated data, and the benchmarking tool and data will not be available to the public. We will send the user group a sample in the next few weeks.

Will the new dashboards show data from previous years’ data collections? Yes. The new Outcomes and Benchmarking dashboards will render all data from Year One (2013-14 data collection)

onward. • The Public Dashboard Report – a PDF report showing the graphs or your published Outcomes data –is only

available for VFA Year Two (2014-15 data collection) ONWARD. To the public, the link to this report will always be for the most recent year.

Can we view the names of benchmarking institutions selected? Currently, the tool does not return a list of the colleges that are in the benchmarking group. AACC is working with

the VFA Oversight Board and the dashboard developers on both the policy and technical changes needed to make this happen. We have receive this request and are working toward fulfilling it.

• Please note that the benchmarking tool will always use selector criterion to establish the colleges in the benchmarking group; colleges will not be able to pick individual colleges against which to benchmark.

Questions &

Answers

Questions During the Webinar (cont.)

Raw File Upload Process

More info about the "raw data" uploadThe Raw Data upload process requires the college to upload 5 files:

1. Student demographics, 2. Student developmental education placement, 3. Student course data, 4. Completions at two years, and 5. Completions at six years.

The data will be extracted, translated and loaded into the VFA database. After the ETL is completed, the data from the five files will be permanently

deleted from the system. The VFA does not keep student-unit record data.

If the raw data upload method will become the "method of choice," will the bulk upload method eventually be abandoned?

The VFA has no plan to abandon any of the three data input methods at this time. We anticipate that many users will move toward the raw upload method because it is a process where data extraction is streamlined and methodology is preserved. Like all data initiatives, though, we understand that users work differently and have preferences for how to enter data.

Questions &

Answers

What data can be downloaded to Excel? All data that you entered for your college can be downloaded to an Excel document,

so that the college may manipulate their own data. Downloading benchmarking data requires more development and is not available for Year Two.

How is the VFA handling the "English/Reading" combination topic going forward? For Year One, the VFA allowed colleges who offered English and reading under a

combined topic to report the progress and outcomes of these students under an English/Reading combination subject. There was confusion and errors in reporting, so for Year Two we ask all colleges who offer the combined subject to report it under the English subject. For these colleges, the reading subject area will be null.

When will the new versions of the VFA Metrics Manual and User Guide be released? There are aspects of the Metrics Manual and User Guide that require all other data

collection development to be completed. Therefore, the VFA Metrics Manual will be released on November 3rd, when data collection opens. The User Guide will be released soon after.

Is there an easy way to report SAM (Student Achievement Measure) using VFA? The data reported for SAM’s associate/certificate model is a roll-up of the VFA’s Six-

Year Outcomes for the full-time and part-time, credential-seeking cohort. If a college reports the VFA, it can easily report SAM. The SAM folks have been working with their developers on a file that helps with getting VFA data into SAM. We will update you on this post-October 20 when SAM launches. See the Resources section of the SAM website for more information: http://www.studentachievementmeasure.org/resources#

Questions &

Answers

Questions During the Webinar (cont.)

Misc.