How Judson ISD Implemented and Tracks IT Metrics & Key Performance Indicators Texas K-12 CTO Clinic, Austin TX June 19th & 20th, 2013 Steve Young CTO, Judson ISD

Key Definitions Metric

Definition: a standard of measurement Examples

Bandwidth available Bandwidth utilized IT cost per student

Key Performance Indicator (KPI) Definition: A set of quantifiable measures that a company or

industry uses to gauge or compare performance in terms of meeting their strategic and operational goals. KPIs vary between companies and industries, depending on their priorities or performance criteria. Also referred to as "key success indicators (KSI)".

KPIs may be different in every organization – and organizational KPIs may be different than department KPIs

Issues at JISD (and many IT departments) • Many IT staff were not aware of data around areas they

supported • If asked, they could not always elaborate on how their

area looked • This means they often could not identify, trends, issues, or

problems • Sometimes data in someone’s silo is useful to other

personnel • Need for increased focus on interacting with users and

measuring the quality of this service • Primarily centered on our help desk

Examples Before KPI…. Q: How many guests are we seeing on the wireless networks

at the high schools? A: I don’t know – we can’t identify this easily Q: What is the trend in our help desk tickets and our ability

to service help desk tickets? A: There are a lot open……

Goals Create an easy to understand visual

representation of metrics and KPIs Make it very simple for staff to update

data from their silos Integrate/automate data analysis from

a few key systems Share this data publicly to let staff (IT & others) know this

is important and to help drive data understanding and improvement

Not to make IT staff mini data clerks and not to turn them into data analysts

Help spark an interest in the data and start conversations about it

What We Did We developed a ColdFusion web application that allows

employees to quickly enter in metrics from their silo Data is displayed in a rotating slide format on web page Most data is historically displayed for identifying trends or

patterns IT staff can pull this up on their desktop at any time But the main way to increase visibility was to place the

rotating slide display on a large digital sign in a public area of our IT department Staff see it several times a day Customers see it when they visit Vendors and suppliers see it

Key IT staff brainstormed metrics that were considered for tracking

Example Screen

Example Screen

Customer Focus KPIs Current KPIs For Customer Focus (from 1 to 5, 1 low)

Target Data Source Person or Auto

Where to Check What are we looking for?

Responsible Parties

Staff Courtesy & Professionalism

4.50 (90%) Auto Help Desk Stats App

Happy Customers, trends

Tech Staff

Staff Understanding of Problem

4.50 (90%) Auto Help Desk Stats App

Happy Customers, trends

Tech Staff

Customer Rating of Response Time

4.50 (90%) Auto Help Desk Stats App

Happy Customers, trends

Tech Staff

Customer Rating of Technician Quality

4.50 (90%) Auto Help Desk Stats App

Happy Customers, trends

Tech Staff

Customer Overall Experience

4.50 (90%) Auto Help Desk Stats App

Happy Customers, trends

Tech Staff

Current Process & System Effectiveness Metrics/KPIs Cycle Time Avg for All Tickets (Weekly)

24 hours

Auto Eduphoria MySQL Shortest cycle time possible Tech Staff

# Tickets Opened (Weekly)

Dir Desktop Eduphoria MySQL Closed to be higher than open, trending up of open tickets / compare to cycle time

Tech Staff

# Tickets Closed (Weekly)

Dir Desktop Eduphoria MySQL Closed to be higher than open, trending up of open tickets / compare to cycle time

Tech Staff

SMTP Server (Monthly) availability

99% Auto Binary Canary API Uptime, service problems Sen Sys Admin

MyJISD Portal (Monthly) Availability

99% Auto Binary Canary API Uptime, service problems Webmaster

Citrix (Monthly) Availability

99% Auto Binary Canary API Uptime, service problems Dsk Svcs Lead

Daily Peak Bandwidth Usage, Daily Avg Peak

<280 Sys Admin LightSpeed Available bandwidth, peaks Dir Network

# of Unplanned Network Outages Per Week

# Help Desk Log issues in Tech Priv Wiki

Recurring & Preventable Issues Dir Network

# of Unplanned App/Server Outages Per Week

# Help Desk Log issues in Tech Priv Wiki

Recurring & Preventable Issues Dir Network

Current Process & System Effectiveness Metrics/KPIs All Scheduled Backups Run Successfully in (WEEKLY)

100% Net Admin HP Data Protect/Unitrends

100% Success Dir Network

% Of Systems with latest A/V Signatures

50% Dsk Svcs Lead Sunbelt VIPRE Console

Too low means machines unprotected

Dir Desktop

# Infected Computers (WEEKLY)

# Dsk Svcs Lead Sunbelt VIPRE Console

Trends indicating infection issues

Dir Desktop

SSRPM Weekly Efficiency (SSRPM Resets / ( SSRPM resets + help desk resets) ) X 100

100 is high - Target 75% (.75) – convert to Percent

Net Admin SSRPM self-reset report and Eduphoria tickets for AD password resets

We want user being able to reset their own passwords – 75% indicates they are doing so. This means less down time and help desk calls.

Dir Desktop, Tech Staff

PEIMS Data Score % Student Data Correct (Certify) (Nightly)

95% Dir Data Svcs Email / Certica Site Low #’s indicate data clerk errors

PEIMS Staff

PEIMS Discipline % Correct (Certify) (Nightly)

90% Dir Data Svcs Email / Certica Site Low #’s Could Indicate Data Clerk Errors or AP errors

Campus Admins

% Cameras Security Cameras Operational (Daily)

100% Spec Sys Admin VideoInsight Software

High % up – spot issues with systems or DVRs

Dir Network

Judson ISD Connect Application Installs

Programmer Conduit Mobile Increased app installs Dev Svcs Team

% of Traditional Campuses Making >1 Outreach Call Per Month

CTO Blackboard Connect Campus to home communication by all traditional campuses

Principals/Asst Supt

% Of Students Being Tracked in Parent Center

Auto SQL Cluster / ParentCenter

Increased access through the year Principals/Asst Supt

# Guests per SSID level (weekly)

Engineer Cisco Prime To measure BYOD adoption – look for increasing – low may indicate technical or admin problem

Dir Network / Principals

# Web Site Visits Webmaster Google Analyt API Look at traffic volume and how servers are handling

Webmaster

# Intranet Visits Webmaster Google Analyt API Look at traffic volume and how servers are handling. Looking to increase visits by staff month over month

Webmaster

eForms Campus Forms Usage & Savings

Auto Eduphoria MySQL Look to see campuses lagging in adoption. Looking for 4 digit form usage per month and for large savings $$

App Coordinator, Principals

Dept sponsored eForms Forms Usage & Savings

Auto Eduphoria MySQL Looking for >1000 forms a month and for large savings $$

App Coordinator, Principals

SPAM Percentage (weekly) Sys Admin Lightspeed Looking to gauge Spam load on servers

Sys Admin

Incoming email (WEEKLY) Sen Sys Admin Exchange Should see normal load week to week. Drop may indicate system or config issue

Sen Sys Admin

Current Process & System Effectiveness Metrics/KPIs

Demo http://www.judsonisd.org/apps/kpi/index.cfm

Impact So Far • There is more awareness about

data • Staff can better verbalize their

numbers • There is more cross function awareness • Increased focus on customer service

• There has been a small increase in this number through the year • Worst ratings are on cycle time

• Good discussions have occurred in front of our “water cooler” KPI sign

• Shows customers we care about data about hopefully shows we care about them

Possible Future Changes • More automated data pulls • More district strategic KPIs • Ability for other departments to track and monitor

metrics important to them • GUI front end for setting up new metrics

Help Desk Phone Calls # Successful Help Desk % Calls Failed

Judson ISD Connect Application Page Views SQL Cluster (TIME FRAME) Availability

Annual IT Staff Retention Rate eSchoolPlus (TIME FRAME) Availability IT Cost Per Student (Annual) / Per Staff FinancePlus (TIME FRAME) Availability

% of Campuses w/ >0 News & Events on Internet

Eduphoria! (TIME FRAME) Availability

Schoolworld website % by campus Phone System (TIME FRAME) Availability

MyJISD daily logins - student and teacher UMRA Student & Employee AD Account Automation %

Lync Chat Volume % of Campus Uptime (just record down time and calc) per campus

Lync Meeting Volume

Possible Future Metrics/KPIs

Other Data – External Benchmarks • Data to compare IT to other districts can be difficult to

obtain • Often it is limited to older PEIMS data • Several Texas and Colorado districts participated in a

comparative benchmarking program from www.Benchmark4Excellence.org

• It is an annual offering with a deep set of data to help see how your organization compares to peers.

• Rather than internal metrics – this focuses on Benchmarks to compare your organization with others around key data elements

• Contact Rick Imig – 281-910-0113, Rick@Benchmark4Excellence.org

How External Benchmarking Helps • The study let me know if an area is on target or way off

the mark compared to peers: • Studies key IT salaries • Studies staff allocation to key IT areas • Much of the data is normalized for cost of living and normalized

across 25,000 students

• The study also compares self assessments on how districts are making progress in key areas in curriculum, maintenance, transportation and other areas: • These are great discussion starters to share with other district

leaders.

Discussion & Closing • Is this really useful? • How would you share this with customers?

• Should or would they care?

• How could this be made better? • What are some things that we should and can measure? • What would be awesome to measure? • How should this be shared with senior leadership?

• Do they care about this data?

• Contact: Steve Young • syoung@judsonisd.org