Sunz2013 mike o_neil

February 2013

Michael O’Neil, Ministry of Social Development

Targeting Public Value in New Zealand

The role of analytics in achieving social, economic, and fiscal outcomes

2 Targeting Public Value in New Zealand

Drivers of Change

• Welfare Working Group recommendations

• Welfare Reform benefit collapse

• Actuarial valuation of long term liability

• Investment Approach


Foundations of Approach

• Identify cohorts of risk

• Model risk

• Trial a variety of interventions

• Measure results

• Continue what works, stop what doesn’t


Analytics as the Enabler

• Analytics is not new in MSD (LLTBR, 6-9 year olds)

• Welfare Working Group gives it traction

• Experimental design techniques

• Enables results measurement

• Demonstrate value


First Efforts

• Aim to identify 15-17 year-old school-leavers most at risk of not being

in Education, Employment or Training

• Model risk of becoming welfare beneficiaries

• Becoming a beneficiary at a young age means higher likelihood of

being a long term recipient

• Changing that pathway generates social, economic, and fiscal

outcomes


Modelling Approach

We're trying to identify 15-17 year-old school-leavers most at risk of not

being in Education, Employment or Training, with an emphasis on

likelihood of long-term benefit liability

'Trigger' Event:

Leaving school

Outcome Window:

3 years

Outcome:

Benefit grant (not UHS)

Client's education history

Client's CYF history

Related person's benefit history

Health, Justice, other Social Sector, etc etc 3-Year Future Liability $

or 'Value at Risk'


Technical Details

• History of benefit uptake on MSD Data Warehouse

• History of interaction with CYF on MSD Data Warehouse

• School Leaver information from Ministry of Education

• Matching client identity data from multiple sources

• Legal and Privacy concerns


Platform


Technical Details

• Electronic transfer of data via B2B infrastructure

• Data Matching using DataFlux Software from SAS

• ETL processes written in SAS

• Campaign Flow built using Campaign Studio from SAS

• Enriched data transferred to ART RDBMS using SAS/Access

• Messages sent to SMS provider using Proc Soap


Technical Challenges

• End to end integration (Ministry of Education through to ART)

• Level of integration of DataFlux with the rest of SAS

• Campaign Software integration with IAP

• Evolution of requirements

• Mixture of Vendor, Contractor, and Permanent Staff

• Time constraint immoveable (legislative imperative)








Data Infrastructure


Where is the Value


Our Role

• Decisions about allocation of resources

• Stewards of public funds (tax payer funded)

• Fundamental duty to optimise allocations (under constraint)

• Analytics (propensity modelling) and Operations Research

optimisation techniques have a major role to play


How Right Can You Be


The Value of Being Right


Common Misconceptions

• People are always better than models

(even gut feel models have a ROC Curve)

• We can have the perfect model

(position on the ROC curve and the cost matrix matter)


Driving Value


Conclusion

• Focus is now on driving value

• Use of analytics combined with optimisation techniques are

appropriate tools in stewardship of public funds

• Platform is very similar to private sector “closed loop” marketing

• Is the genie out of the bottle?


Acknowledgements

• James Mansell – Innovation and Change Leadership

• Evan Stubbs – Though leadership

• John O’Leary – A rare breed of Account Manager

• Wessel de Meyer – Outstanding Diagrams

• IAP Team – Anarchic response to being managed

• SAS Institute NZ - Patience

Documents

Sunz2013 mike o_neil