Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
CRICOS 00111D
TOID 3069
Deriving value from data and analyticsPerspectives from Swinburne's journey
6th August 2019
• Centrally located team supporting
the whole university
• Deep technical expertise
• AWS & Tableau certified
Swinburne
Information Hub
(SIH)
Student One
Government
submissions
Executive reporting
Management reporting
Exploratory reporting
Dynamic ad-hoc
Student One - Operationalreporting
Governmentreporting
Ag
gre
gate
dTra
nsa
cti
on
al
An
aly
tical
Analytics University Wideanalytical models
Business Analytics
Executive and management reporting, student operational reporting, data-related submissions to government and
maturing analytics to support decisions.
Harnessing the capabilities of a unique team
• No data repository, leveraging replicated data files
• Requests unmanageable, unable to keep up
• Between a dumping ground and a graveyard
• Tech-specific skill set
Issues across the organisation
Issues within the business analytics team
• Multiple data views creating confusion and conflict
• No single point of reference for University
• Lack of trust in reporting produced
• Increasing need to be faster
In 2016 we commenced a data and analytics project to address concerns around the accessibility and reliability of
reporting from siloed data sources, and which sought to lay the foundations for analytics to provide decision support.
What was our starting point?
• Driven by speed and cost we ran a
PoC on AWS, demonstrating the
value of security, encryption and
performance
• AWS was chosen after taking into
account aspects such as cost,
reliability, security and data
sovereignty considerations
• AWS also provides the ability to
readily expand in terms of capacity
and services as business demand
requires, i.e. data lake, NLP, voice
navigation, AI, machine learning
• Business Analytics moves the data
from S3 to Redshift, modelling within
Redshift, building data sets and
dashboard development in Tableau
and managing/training usersAWS - ap-southeast-2 (Asia Pacific (Sydney))
VPC contents
VPC subnet
Swinburne data center
Internet gateway
virtual private gateway
routerVPN
connectioncustomer gateway
Amazon S3
client
Internal Sources
Internet gateway ESB
firewall
encrypted data
encrypted data
SSL Connection
SSL Connection
Amazon Redshift – Dev/
UAT
Amazon Redshift - Prod
Amazon EMR
To provision insightful information, we needed to develop a data repository and introduce tools to effectively manage the
data; current AWS architecture started as a POC and has grown into the enterprise management reporting platform.
Proof of concept to enterprise repository
• Picked the pieces of Data Governance that
we needed to drive trust and consistency in
the data
• Took on Data Standards and Data Quality
first. Simple form and publish approach –
wiki for access
• Great tool to drive business involvement
and for them to understand their role and
how that drives success
• Utilised a working group for business
representation, got volunteers to own data
sets and ran forums for updates and as
motivation discussions
• Simple and doable for a small team - now
embedding in whatever we build and
deliver to the business
A lack of data governance was at the core of the distrust of data and reporting; we chose to work within what we could
control by focusing exclusively on the data for the platform.
Data standards & data quality
• Re-launched and put
ourselves out there
• Created a brand –
DAPPER
• Deployed a host of
techniques to
engage including
journey wall, tours,
LAB, lunch & learn
and video
• Engaged Executives
first – 1:1 training,
buy in and quotes
For change to be effective the biggest hurdle is getting people to believe in and trust the data; so we focused on building
a brand that became synonymous with the single source of truth rather than invest in tools or focus on the technology.
Change the what, how and why?
Lead
Certification
Dapster
Dynamic
Designer
Enterprise
Dashboards
DownloadData sets
BYO data to SIH
Views FiltersSimple reports
Use Intro
Save as
Publishto folder
Web edit
Desktop
Contribute
Responsibility
Publish onDAPPER
Building
New
cap
abili
ty
New
leve
ls o
f ac
cess
& c
apab
ility
Building our data maturity and capability across the
University:
• Continue developing end user capabilities
• Adding new levels of accessibility
• Deal with potential risks associated with increased
data availability
• Understanding Swinburne’s Executive and Management
reporting needs
• Make more information accessible to users
174 reports, 800+ trained users with >15,000 report views annually with a continued focus on extending the accessibility
and functionality of the services and training provided.
DAPPER is utilised across the University…
Navigation –
Landing Page
Scorecards - Swinburne
on a Page
Analysis Dashboards –
Management Reporting
Filter Based Reports –
Exploratory Reports
Build Templates –
Web Edit Functionality
As data maturity has increased, we are extending the utility of what has been built by allowing selected users to download
data, to augment with their own data not available yet hosted and/or to publish their own content.
…with information consumed in many ways
Information that previously took enormous effort to manually update and email to participants is now discussed in
interactive online meetings by sharing filtered views that are refreshed daily, in a restricted and secure fashion.
Value of sharing insights interactively
• Committee meets regularly prior to intake periods to
evaluate marketing and recruitment activities
• Restricted access controls who can see commercially
sensitive data
• Online meetings, with users interacting with and
sharing filtered views refreshed daily in production
• Prior to DAPPER, spreadsheets created manually
across various teams each week and sent via email
This rich information is now available to all parts of the business for activities such as strategic planning and
benchmarking, marketing, product development and rationalisation decisions and forecasting future enrolments.
Value of data democratisation
• HEIMS reports
identify market
opportunities,
bringing together
comparative
enrolments
benchmarked
against Australian
universities
• No prior reporting
available - the
business sourced
information
manually c/o
publications and
websites with no
enrolment counts
Building out an enterprise reporting capability with robust data management and governance has enabled significant
improvements in both the efficiency to create reports and their timely delivery to those that benefit from the insights.
Value of delivery efficiency
• Previously took >1wk from raw QILT
data to Tableau reports; now takes
~30min to upload data that’s
available immediately
• Austrade data previously
significantly underutilised, was
delivered in Excel format making it
hard to work with
Foundational work and further modelling permits us to identify at-risk students based on >400 attributes, working with
academic and engagement teams to provide targeted intervention and support to improve student retention.
Attrition propensity modelling
StudentLife
Video &Echo 360
CRM(B2C)
Canvas LMS& Blackboard
Transition?
SurveysGOS, SES,
SFS, Checkin
Commute
Interventioncohort 1
Interventioncohort 2
Interventioncohort 3
Contact and record outcomes
Statistical appraisal of efficacy
Report outcomes & pass embed learnings into future interventions
Reinforcementlearning
Existing attrition propensity model Student-on-a-page (incl. attrition risk) in CRM Leverage Audience/Campaign in DXP
AND/OR campaigns direct from
CRM
Campaigns routed via
DXP
New data will greatly enrich modelling accuracy
• Increased accuracy through machine learning
• Shortened timelines; 2019 & 2020 views 15th April
• Greatly reduced staff input initially
• Provide 3 NSE scenarios; weighted YoY growth, YTD
run home extrapolation and ML models
• NSE & RSE linked, generating 5yr total EFTSL views
99.58%
99.41%
99.21%
99.58%
99.49%
DecisionTreeRegressor
RandomForestRegressor
GradientBoostingRegressor
ExtraTreeRegressor
Ensemble
• Initial view of EFTSL/SCH and gross load revenue
forecasts for all HE and VE units for both NSE and RSE
students for refinement by marketing, faculty & finance
• Mechanism to capture adjustments & visualise
commensurate impact, passing final EFTSL/SCH data
back for ingestion into GL and to facilities & timetabling
• Method and logic fully documented on Wiki, budget
versions stored for roll-back and comparison
The use of machine learning and systemised capture and visualisation of adjustments allows us to develop initial views of
student load forecasts earlier with greater accuracy and less manual intervention.
Forecasting and load planning
𝑚𝑖𝑛
𝑖
𝑛
𝑥𝑖ℎ𝑖𝑐𝑖𝑤𝑖
Constraint optimisation (cost function)
𝑥𝑖 is number of unallocated class instances
ℎ𝑖 is duration of unallocated class
𝑐𝑖 is hourly cost of sessional staff for the class activity
𝑤𝑖 is the number of weeks the class runs
ABC10001
Tutorial 1/01
Monday
10:30 AM
AKD60001
Tutorial 1/01
Wednesday
9:30 AM
XBC30020
Lab 1/03
Friday
9:30 AM
ABC10001
Lecture 1/01
Tuesday
2:30 PM
XBC30020
Lab 1/02
Thursday
12:30 PM
AKD10001
Lecture 1/01
Friday
2:30 PM
XBC30020
Lab 1/01
Monday
5:30 PM
ABC10001
Tutorial 1/02
Tuesday
4:30 PM
ABC10001
Tutorial 1/03
Wednesday
3:30 PM
ABC10001
Tutorial 1/04
Thursday
4:30 PM
XBC30020
Lecture /01
Monday
2:30 PM
AKD60001
Tutorial 1/02
Wednesday
4:30 PM
1
2
3
4
5
6
7
8
9
10
11
12
Timetable
10 45 61 23 7 8 911 12
XXX60009 – Tutorial1…
AHC20001 – Lab1AHC10004 – Tutorial1
ABC10001 – Tutorial1
222222 20 18
223024 28
Enrolment headcount : 104
Max class size : 32
Existing 5 instances per week
Proposed 4 instances per week
Duration : 2 hours
Teaching period :12 weeks
Potentially free up 24 delivery hours
45 61 2 37 89 1011 12
Using unit EFTSL forecasts to identify potentially over/under capacity unit instances and provide an initial Academic
Workload Model view maximising utilisation of allocable academic teaching hours.
Higher Education resource modelling
Bringing together forecast student load and initial views of resourcing to develop a view of margin as part of a broader
suite of metrics to appraise product performance.
Forecast labour delivery margin
• Unit enrolment
projections from
the forecast
(EFTSL and $)
• Optimised
allocation of
academic and
sessional
resources using
prior years actual
unit instance
profile
• Equals unit
delivery margin
(gross load
revenue less
directly attributable
labour)
• User-driven, with
adjustments to EFTSL
triggering recalculation
• Fast turn-around, with
E2E refresh ~7 minutes
• Removes bottlenecks,
with users able to load
adjustments and
access outputs directly
• Detailed visualisations
for each step
(fcst/budget, resourcing
and delivery margin)
• Ability to flex aspects
makes more dynamic
• Versions archived,
allowing roll-back
• Plan to link tightly to
facilities & timetabling
• EBA
• Research allocation
• Capability matrix
• Salaries & sessional rates
• Room/capacity
• Instances (planned)
• FOE rates & pricing
• NSE EFTSL @ nFOE
Linking load forecasting, resourcing & delivery margin for end-to-end scenario modelling capability to support faculty
planning, improving alignment between anticipated enrolments and scheduling, with refreshes processed in ~7 minutes.
HE resourcing budget initiative (HERBI)
1,450
925
2,375h
month
99,130h
Team
51,840h
year
Survey of 230 users before
we developed DAPPER
Extrapolating effort per annum
across 800+ Dapsters less the
total resources in our team
Data prep
Reporting
~6% month
Through the provision of a centralised, governed reporting and analytics service, even saving 50% of the time people
traditionally spent sourcing, preparing and reporting on disparate data provides significant value.
Realising the value; estimated to exceed $3m
Example only
Developed to uniformly support new product development, accreditation, KRA reporting, faculty planning & strategic
quarterly review requirements, bringing together >40 metrics from 12 internal and external data sources spanning ~8m rows.
Consolidated governed product datasets
• Consolidating data
provides a means to
appraise and
benchmark course
performance and
identify new product
opportunities
• Improves consistency,
with single view of
metrics across internal
& external sources
• Allow users to access
and utilise this
information self-
sufficiently to reduce
bottlenecks and
facilitate ad-hoc
exploration and
analytics
• Using data from our learning
management system (LMS)
we provide insights into
assignments due dates
across multiple units and the
weighting applied to each
assignment
• This provides an overview
schedule of student workload
& contribution to final grades
across the semester
• This view can be used to;
• Optimise academic
resourcing by identifying
peak demand
• Provide a better student
experience by
distributing assignments
more evenly
Assignment schedule planning and weighting analytics provides opportunities to optimise resourcing and balance
academic demands on students, improving the experience for both.
Canvas LMS reporting and learning analytics
FUTURE READY
LEARNERS
Founded at Swinburne in Apr’18 focused on FRLs, our new data consultancy works with talented placement
students to develop skills in demand…data management, visualisation & analytics.
The Data Experience · Giving students the EDGE
Learning by Doing Our Partners
ENGAGEMENT
o Business Consulting
o Change Management
o Training
PERSONAL
o Personal Development
o Personal Branding
o Communication
BUSINESS
o Direction and Ask
o Business Operations
o Data Governance
DESIGN
o Facilitation
o Visualisations
o Data Management
TECHNOLOGY
o AWS
o Tableau
o ThoughtSpot
Our 1st cohort exceeded expectations – mastering bootcamp, delivering Swinburne projects, achieving
industry-recognised certifications, undertaking paid industry engagements and receiving employment offers.
The Data Experience · Program overview
Bootcamp Internal Projects Internal ProjectsPaid Industry
Experience
Part-time
employment
Tableau
Certification
AWS
CertificationAttend Tech
Conference
Client
Presentations
“Working externally has enabled me to
develop and enhance the skills I have learnt
during my time in higher education“
“After a year of practical
work experience, I better
understand the direction I want my career to take.”
“Fast, fun learning.”
“A great way to
recognise the
importance of soft
skills and areas for
development.”
AWS CIC
or
Public
Sector
Challenges
ENGAGEMENT
PERSONAL
BUSINESS
DESIGN
TECHNOLOGY
AWS
Medical Insurance
CompanyEnergy
Company
CALIFORNIA
ARIZONA
BUSAN
MELBOURNE
MUNICHBERLIN
FRANCE
GLOBALLY CONNECTED
The Swinburne Data for Social Good Cloud Innovation Centre powered by Amazon Web Services is the 1st in the Southern
Hemisphere and 8th globally, using applied research to provide a digital transformation capability for Government and NFPs.
Swinburne Data for Social Good
• Matt Rudd
• Email [email protected]
• +61 409 346 675
https://www.thedataexperience.com.au/
http://swinburne.edu.au/dataforsocialgoodcic/
Questions or more information