Business Case Development and IT Project Oversight in the Government Environment NASACT Middle Management Conference April 13, 2011 Presented by: Sean

Business Case Development and IT Project Oversight

in the Government Environment

NASACT Middle Management Conference

April 13, 2011

Presented by: Sean McSpaden, Deputy State Chief Information Officer

2

Table of Contents

IT Project Performance (across the nation) IT Controls & Oversight Framework IT Investment Lifecycle Diagram Proposed IT Projects 09-11 & 11-13 IT Investment Review and Approval Policy Quality Assurance (QA) Oversight Policy IT Standards (Controls & Oversight) Resources & Contact Information

3

IT Project Performance

Public and private sector organizations across the nation have had significant challenges in meeting originally stated budget, schedule and quality objectives for large IT projects.

2002 Article - MIT’s Sloan Management Review Estimated that 68% of corporate IT projects were neither on time or on

budget, and didn’t deliver on originally stated business goals and objectives

2004 Computerworld Article “…72% of large projects are late, over budget or don’t deliver

anticipated value…a 28% chance of success.” Standish Group (2004)

Studied over 40,000 projects in 10 years to reach the findings Project success rates increased to 34 percent of all projects. More than

a 100-percent improvement from the success rate found in the first study in 1994

4


Standish Group International - 2001

5


Gartner, Inc (“Exploring the Relationship Between Project Size and Success - 2008) Not only are large projects more likely to fail than small projects, but

cancellations of large projects occur at a later point in the project life cycle, thus incurring huge costs

Two-thirds of the canceled projects with budgets exceeding $1 million were canceled when they were more than 50% complete, while cancellation of midsize projects typically occurred prior to reaching 50% completion.

6


Gartner, Inc (“Why IT Projects Fail in Government” – 2006) Top 10 Reasons Why IT Projects in Government Fail

Unclear or unrealistic business case Misaligned accountability and incentive structure Insufficient management or technical expertise by the external

service provider or unfamiliarity with the agency's or government's architecture

Poor project discipline and process controls that impede the ability to make informed decisions

Inadequate performance management practices and tracking systems

Ineffective governance Uncertain budget environments Failure to define, control and track changing requirements External factors such as change of administrations, excessive or

intrusive oversight, and external service provider mergers or bankruptcies

Government and external service provider overconfidence as to risk

7


IT projects surveyed by the Standish Group in 2009 showed a “marked decrease” in project success rates.

Nearly 70 % of IT projects were deemed “challenged” or were failed projects that were either cancelled or were delivered and never used. Specifically, 24% failed, i.e. canceled or work products never used 44% were deemed challenged, i.e. late, over budget, and/or delivered

work products with less functionality than promised; 32% were deemed successful, i.e. on time, on budget, and work

products fully functional. Oregon state agencies have carried out many major IT projects in

support of agency business over the past decade…also with mixed success.

8

CNIC Assessments & Findings

Computing and Networking Infrastructure Consolidation (CNIC) Project Three (3) third party assessments performed in 2006

Secretary of State Audit (Report No – 2006-33) Quality Plus Engineering (hired by Legislative Fiscal Office) Solutions Consulting, Ltd. (Quality Assurance Contractor)

Findings – State did not have sufficient IT Governance Financial and Business Case Analysis Management Controls Architecture and Standards Quality Assurance Processes IT policies and procedures Management and Technical Expertise

Lacking remediation.. the undertaking of enterprise level, large scale IT projects is at substantial risk.

9

IT Project Risks

Large IT projects that span multiple years are inherently risky and complex.

Large IT projects (with few exceptions) exceed $1M and span multiple years, sometimes multiple biennia, in duration.

Original budget and schedule estimates for these projects were, in most cases, established twelve to fifteen months prior to the biennium in which the agency plans to initiate the project

Large IT projects require a control structure and the consistent application of controls for scoping, planning and executing work

Changes or variances in scope, quality, schedule or budget, should be monitored and root cause corrected

Risk controls should anticipate variances and mitigate them through planned alternative strategies

Objective: ‘management by exception’ not ‘management by crisis’

10

IT Investment LifecycleDiagram

Business & Technology

Planning

Budget Development & Approval

Detailed Project

PlanningProcurement

Project Execution,

Implementation & Closing

Biennium Boundary

Conduct IT Investment Review Provide Recommendations to BAM/LFO

High Level Project

Planning & Business

Case Development

Quality Assurance Oversight & Spot Audits LAB Approved Projects – Project information provided met all IT Investment Review and Approval Requirements

IT Investment Review and Approval Process1) Projects that lacked sufficient detail in the budget process2) Off-budget cycle projects

Assess SDC Impact & Architecture Alignment


Agency

DAS EISPD & SDC

Post LAB - Business Case & Project Plan Reconciliation

* EISPD – Enterprise Information Strategy and Policy Division* LAB – Legislatively Approved Budget* SDC – State Data Center

“On-Budget” Cycle “Off-Budget” Cycle

11

IT Controls Framework

Governance Since 2007, established governance charters for the State Data Center

(SDC) Advisory Board, SDC CIO Advisory Board, CIO Council, and CIO Management Council

Agencies with Major IT Projects required to form steering committees

Enterprise IT Planning Enterprise Strategy adopted in 2007 and updated in 2010. Enterprise Security Plan adopted in 2009 Enterprise GIS Strategy completed in September 2010 E-Government Transition strategy completed January 2010

IT Budget Instructions – Biennial Budget Development process Developed Biennial Budget instructions requiring collaborative planning

between the DAS State Data Center and its primary customer agencies, and the creation of business cases for major IT projects.

Provided agencies with IRM Planning Guidance Provided agencies with IT Lifecycle planning guidance and templates

12

2011 – 2013Agency IT Budget Instructions

Requirements (All Agencies)

• IT Project list for projects >$150,000 (Policy Option Package (POP) or Base)– Budget Form (107BF14)

• “Major” IT Projects >$1,000,000 (POP or Base)– Budget Form (107BF14) – Business Case Document

• Establish standard lifecycles for agency IT assets and develop and submit lifecycle replacement plans

– Required by State IT Asset Inventory and Management Policy– Sample plans provided on request

Requirements (SDC Customer Agencies)• SDC involvement in IT project planning and budget development prior to

agency budget submission to DAS Budget and Management• Informational Websites:

http://www.oregon.gov/DAS/EISPD/ITIP/IT_Budget.shtml http://www.oregon.gov/DAS/EISPD/ITIP/IT_Lifecycle_Planning.shtml http://www.oregon.gov/DAS/EISPD/Business_Case.shtml

Note: Helps fulfill agency and DAS IT Portfolio Management-related statutory obligations (ORS 184.473-184.477)

13

Business Case Development

Business Case Development Since May 2007 over 300 people have completed business case

training During the budget development process - Business cases are

required for all projects that exceed $1M Prior to execution - Business cases (new or refreshed) are required

for projects that exceed $150,000 per the current IT Investment Review and Approval Policy

For all Major IT Projects (POP or Base >= $1M) agencies required to submit a business case document that clearly describes how the project/initiative:

• Aligns with and supports agency strategic/business plans• Aligns with and support the Governor’s goals, priorities and

initiatives, the Enterprise Information Resources Management Strategy, and other IT-related statewide plans, initiatives, goals and objectives.

14

Business Case Development

The business case should also include the following information:

• Subject, Purpose & Scope• Projected cash flows across timeline (lifecycle or other) • Alternatives Analysis (to the extent possible at this point in the project

lifecycle)• Assumptions & Methods that the investment is based on • Costs & benefits – Financial & Non-financial (to the extent possible at

this point in the project lifecycle)• Critical Success Factors • Risk Assessment (to the extent possible at this point in the project

lifecycle)

Business case development resources can be found at: http://www.oregon.gov/DAS/EISPD/Business_Case.shtml

15


Architecture and Standards Development Since October 2007, provided Enterprise Architecture Development

training (TOGAF) to nearly fifty (50) state staff Architecture development work in progress at State Data Center and

within several agencies (DOR, Employment, DHS, ODOT, DAS, Forestry)

GIS Software Standard, GIS Data Standards, Email Server Software Standard, and Enterprise Security Architecture and Standards adopted

2008 - Revised IT Asset Inventory and Management Policy and conducted asset inventories in 2008, 2009 & 2010

IT Standards Website established http://www.oregon.gov/DAS/EISPD/ITIP/Standards.shtml

16


Project Management Training (1997-Present) Over 300 state and local government professionals successfully completed the

Oregon Project Management Certification Program (OPMCP) since March 2007 Over 900 people have completed the program since 1997

Established Oregon Project/Portfolio Management Advisory Board – 2010 Champion the use of project managers and project/portfolio management practices

in state government.

Identify or define project/portfolio management best practices and standards, and promote them in collaboration with all state agencies.

Recommend new or revised project/portfolio management policies to Governor’s Office, Department of Administrative Services (DAS), and/or state agencies.

Provide and oversee the training of state employees in project/portfolio management practices and techniques. The Board’s training oversight may also include the development of a portfolio management certification program.

Define qualifications, standards and certification requirements of OPMCP

Work with DAS on project manager job classification specifications, minimum qualifications, recruitment, and retention issues

17


Quality Assurance

All Major IT projects are required to have third party quality assurance oversight and submit quarterly reports to DAS per the State’s Quality Assurance Policy

March 2009 - Contracts with 11 QA firms put in place

Consistent Statement of Work, Standardized reporting templates and Quality Standards Checklists in place

Lessons Learned

2009/10 - Established Lessons Learned Website

2010 - Require Lessons Learned reports for every reviewed project -

2011 - Holding web conference calls/meetings to share lessons learned on various topics (procurement, planning, oversight, etc.)

18

Quality Assurance Reporting

19


Statewide IT Training Contracts – February 2009 Training to be provided across six categories

Management (e.g. Change Mgt., BCP, ITIL, COBIT) Infrastructure (e.g. Network, OS, Firewalls, Security) Application Development (e.g. Java, Visual Basic, XML) Database Management (e.g. Oracle, SQL, DB2) Technical Support Services (e.g. Helpdesk, LAN/Desktop) Use of Information as an Asset (e.g. Data Mgt., GIS, ERP)

Contracts were executed in February 2009 with four vendors Crossvale, Netdesk, Touchstone, and Webage

Continue to provide agency access to technical resources via the IT Managed Services Provider contract Staff Augmentation (Broad set of skilled resources) Deliverables – based work order contracts

20


Much Work Remains to be Done

21

State IT Project Requests2009-11 LAB

Project Requests By Dollar Amount - >$1 M

AgencyNumber of IT Project

Requests Total Funds for all IT Project Requests

Human Services 9 $104,387,560

Transportation 4 $6,656,000

Education 2 $10,692,400

Administrative Services 3 $6,731,829

State Police (OWIN) 1 $191,695,000

Judicial Department (e-Court) 1 $20,345,000

Totals 20 $340,507,789

Sample projects included in LAB— DHS Behavioral Health Integration

Project— DHS OR-Kids (Child Welfare Information

System)— Education – KIDS Integrated Data

System— Education - OVSD - Oregon Virtual

School District— DAS Enterprise Learning Management

System— DCBS E-Permitting Project

Sample delayed or cancelled projects – not included in GRB/LAB

— DAS Human Resource Information System Project

— ODOT Enterprise Resource Planning Project

— DAS Enterprise Architecture and Standards Program

22

Major IT Project Portfolio2007 & 2008 Completed Projects

Thirteen (13) Major IT Projects (Completed in 2007 & 2008) Projects by Agency Budget ($) Project Start CompletedAdministrative Services – 2 Projects

Computing and Networking Infrastructure Consolidation (CNIC) $44.1 M March 2004 July 2007

Oregon Purchasing Information Network (ORPIN) – Release 2 $3.3 M June 2005 December 2008

Agriculture – 1 Project Pesticide Use Reporting System (PURS) $1.9 M January 2006 January 2007

Corrections - 1 Project Corrections Information System (CIS) Rewrite Phase 2 (Project Closed) $4.7 M July 2007 May 2008

Education - 1 Project Pre Kindergarten through Grade 16 Integrated Data System (KIDS) – Phase 2 $2.5 M February 2006 April 2007

Environmental Quality – 1 Project Air Contaminant Source Information System (ACSIS) and Integrated Compliance and Enforcement module (ICE) Application Re-engineering

$1.5 M June 2005 January 2007

Fish and Wildlife – 1 Project Point of Sale (POS) Replacement Project $0.6 M November 2005 August 2007Human Services – 3 Projects Electronic Death Registration System (EDRS) $2.9 M April 2005 March 2008Electronic Birth Registration System (EBRS) $2.4 M December 2006 August 2008Medicaid Management Information System (MMIS) $80.7 M July 2000 December 2008State Police – 1 Project Oregon Wireless Interoperability Network (OWIN) – Phase I Design and Engineering $1 M January 2006 January 2007

Transportation – 2 Projects Right of Way Data Management System – Release 1.0 $3 M April 2005 April 2007Regional Trip Planner - Release 1.0 $2.3 M July 2002 January 2007

23

Major IT Project Portfolio2009 & 2010 Completed Projects

Eight (8) Major IT Projects (Completed or Closed in 2009-2010)

Projects by Agency Budget ($) Project Start Completed

Administrative Services – 1 Project

Enterprise Information Security $14.6 M January 2005 January 2009

Education - 1 Project

KIDS III Project (Pre-Kindergarten through Grade 12 Integrated Data Systems Project)

$7.2 M October 2007 January 2010

Oregon Liquor Control Commission – 1 Project

Licensing, POS, Merchant Business (POP 301) $3.6 M February 2006

Closed - Limitation

Removed March 2009

Transportation – 5 Projects

Transportation Operation Center (TOC) – Event Management $5.4 M December

2003 September

2009

ODOT – DMV Driver License Issuance (DLI) (a.k.a. Real ID) $ 3.7 M

November 2005

July 2010

Commercial Drivers License/Problem Driver Pointer System (CDLIS/PDPS) Release 3

$3.1 M October 2005 December 2010

24

Major IT Project PortfolioCurrent – February 2011

Ten (10) Major IT Projects (as of February 2011)

Current Projects by Agency Budget Estimate ($) Project Start Est. Completion

Consumer and Business Services – 1 Project

Statewide ePermitting – Phase 1 $12,817,343 July 2007 February 2011

Human Services – 4 Projects

Behavioral Health Integration Project (BHIP) $ 25,889,354 October 2007 June 2013

Immunization Information System (IIS) $ 2,054,522 July 2007 May 2011

Oregon Kids (OR-Kids) $ 68,589,233 January 2005 Under Review

CAF-Self Sufficiency Modernization (CAF-SSM) Program $12,750,000 September 2008 June 2011

Public Employees Retirement System – 1 Projects

RIMS Conversion Program – Phase 2 (RIMS/ORION) $39,651,232 May 2005 July 2011

Transportation – 4 Projects

ODOT Right of Way Information Tracking System (RITS) $5,000,000 January 2008 June 2011

ODOT TransInfo Project (TransInfo) $4,225,000 July 2007 April 2011

ODOT DMV Automated Testing Devices (ATD) $1,475,000 September 2007 June 2011

ODOT DMV Microfilm Replacement (MR) $1,173,858 February 2010 June 2011

$173,625,542

25

Major IT Project PortfolioTo be added in Near Future

Twelve (12) Major IT Projects to be added to Portfolio in future reporting periods (as of February 2011)

Future Projects by Agency Budget Estimate ($) Est. Start Est. Completion

Administrative Services – 1 Project eGov Program Transition Budget & Schedule in Development (RFP Negotiations) Employment – 3 Projects Identity and Access Management $ 2,306,988 Under Review Electronic Document Management $ 6,736,013 Under Review Electronic Data Warehouse $ 1,896,695 Under Review Human Services – 2 Projects

Prescription Drug Monitoring Program $ 1,600,000 Under Review

Integrated Collection Management (ICM) Project $ 2,552,172 Deferred to 11-13

Revenue - 1 Project

Revenue Transformation Project $ 90,209,000 Under Review

Oregon State Police – 2 Projects

Computer Aided Dispatch (CAD) Replacement $ 2,268,237 Under Review

Records Management System Replacement $ 1,489,000 Under Review

Transportation – 3 Projects ODOT Oregon Wireless Interoperability Network (OWIN) Under Review ODOT DMV Commercial Driver License Information System (CDLIS) Modernization $ 796,580 September 2010 February 2012

ODOT Expanded Customer Numbers $ 3,440,772 October 2010 February 2015

$ 113,295,457

26

IT Investment LifecycleDiagram


Planning


Detailed Project

PlanningProcurement

Project Execution,


Biennium Boundary


High Level Project

Planning & Business

Case Development





Agency

DAS EISPD & SDC



“On-Budget” Cycle “Off-Budget” Cycle

27

IT Investment Review/ApprovalStatutory and Policy Framework

Oregon Revised Statutes ORS 184.473-184.477 - IT Portfolio Management ORS 283.505 – 283.510 – Acquisition/coordination of telecommunications systems ORS 291.038 – State Agency IT planning, acquisition, installation and use Additional statutory guidance - ORS 184.305, 184.340, 283.140, 283.500, 291.018,

291.037, 291.047, 293.595 Executive Orders: 01-25, 00-02, 99-05, 98-05 Note: All acquisitions are subject to Department of Justice legal sufficiency and Department of

Administrative Services purchasing rules

Statewide Policy IT Investment Review and Approval (April 2010) Technology Strategy Development & Quality Assurance Reviews (Feb 2004)

ITIP Policy URL: http://www.oregon.gov/DAS/EISPD/ITIP/pol_index.shtml IT Investment Review and Approval Policy: http://www.oregon.gov/DAS/EISPD/docs/107-004-130.pdf

28

IT Investment Review/Approval

Policy Purpose – to ensure that state agency IT investments are: Aligned with governor’s priorities and state enterprise IT goals,

objectives and strategies Justified by sound business cases and linked to agency business plans Effectively and efficiently managed utilizing appropriate system

development lifecycle, project management, and quality assurance methodologies

Assessed for financial, organizational and technical risk Pursued after agency business processes have been thoroughly

analyzed (and reengineered, if appropriate). Process analysis and reengineering should occur prior to automation.

Leveraged to the maximum extent reasonable for the benefit of the enterprise. Opportunities for partnering with other agencies or jurisdictions should be explored prior to project initiation.

Clearly documented so that necessary information about such investments is centrally cataloged for information sharing, reporting, and planning purposes

29


Initial review and approval of IT projects involving acquisition (s) > $150,000

In support of SDC, Information Security, and GIS Initiatives, EISPD performs 100% review regardless of dollar amount of:

Mainframe, Midrange, Server hardware IT Security hardware, software, and services Non-ESRI GIS Software and Services

Agencies must complete an Information Resources Request (IRR) and Business Case/Feasibility Statement

Sixty (60) IRRs were submitted since July 2009. More rigorous business case development and risk assessment is

required for larger investment requests Recommendations regarding approval or denial of the request, and

ongoing QA oversight requirements are given to State CIO for final decision

30


Process Diagram A

ge

nc

yD

AS

EIS

PD

Sta

te C

IOS

tate

P

rocu

rem

en

t O

ffic

e

Agency Prepares IRR

Form

Agency Submits IRR

Form

EISPD Receives IRR

Form

Required info submitted?

No

Review/Analyze IRR &

other infoYes

Agency Reviews &

Approves IRR

DA

S S

ME

’s

Analyze requests & Provide

Recommendations

Request for review R

eco

mm

-e

nd

atio

ns

Respond to EISPD Requests for Additional Info

Add. Info. Required

Yes

Recomm-endation

Discussion

No

Receive EISPD Recommendation

Approve/Deny IRR

Agency receives IRR Approval/

Denial*

Receive State CIO Decision/

Forward on to Agency

Decision

IRR Approval/Denial

Forward Approved IRR w/

Procurement Docs to DAS/

SPO

Receive Procurement

Docs w/approved IRR from Agency

Complete procurement/Notify EISPD when Contract Executed

Received SPO Notification of

Contract Execution

Receive Agency Lesson’s Learned

Reports (if applicable)

Complete Procurement Process & Execute

Contract w/DAS SPO

* If CIO denies IRR, Agency may decide to cancel project, repeat entire process or reenter process at appropriate place.

Complete Procurement Process w/

agency

Complete Project/Submit Lessons

Learned Report to EISPD (If required)

31

IT Investment LifecycleQuality Assurance Oversight


Planning


Detailed Project

PlanningProcurement

Project Execution,


Biennium Boundary


High Level Project

Planning & Business

Case Development





Agency

DAS EISPD & SDC



32

Quality Assurance Oversight

Statutory Authority: 184.475, 184.477, 291.037, 291.038 Current Policy – February 2004

Objective: Ensure successful implementation of major IT projects

Defines planning and oversight expectations for different project categories

Tier 1 – Strategic IT Investments - > $5 M Tier 2 - $1 M - $5 M Tier 3 - < $1 M

Ensures QA program resources, executive sponsorship, and project management discipline are applied throughout the entire IT Investment Management Lifecycle

Technology Investment Strategy Development & QA Reviews Policy http://www.oregon.gov/DAS/EISPD/ITIP/docs/QAPolicy107004030Final_posted_20040312.pdf

33

Quality Assurance Oversight

Program leadership: Deputy State Chief Information Officer Methods

Regular assessments performed by independent third party QA contractors Direct participation on project steering committees Project status interviews with project managers and QA contractors

Major IT project Reporting – Primary Focus: Tier 1 & 2 State’s most strategic/critical IT investments

2010 - 2011 Quarterly Reporting February 2010: 12 projects – overall portfolio value exceeds $167 M May 2010: 11 projects – overall portfolio value exceeds $160M August 2010: 12 projects – overall portfolio value exceeds $170 M November 2010: 11 projects – overall portfolio value exceeds $170 M February 2011: 13 projects – overall portfolio value exceeds $180M

Current investment values range from approximately $1.2 M for the ODOT DMV Microfilm Replacement Project to ~ $68 M for the DHS Oregon Kids (OR-KIDS – formerly SACWIS) Project.

34

Quality Assurance Reporting

35

Governance Methodologiesand Standards

Methodology Standards Project Management

Project Management Body of Knowledge (PMBOK) Since 1997 – Over 900 people have completed the Oregon Project

Management certification program IT Service Management

IT Infrastructure Library (ITIL) Adopted by the SDC and several large agencies

IT Security ISO 27001, ISO 27002 Required by Enterprise Security Office and used by SOS for Information

Security Audits Control Objectives for Information Technology (COBIT)

Utilized as SOS audit standard Required by State Controller’s Division for management control of financial

systems

Other – To be determined

36

Questions/Comments?

37

Resources

IT Investment Review and Approval Policy http://www.oregon.gov/DAS/EISPD/IRR.shtml http://www.oregon.gov/DAS/EISPD/docs/107-004-130.pdf

Technology Investment Strategy Development & QA Reviews Policy

http://www.oregon.gov/DAS/EISPD/ITIP/docs/QAPolicy107004030Final_posted_20040312.pdf

Note: Policy is scheduled for revision in 2011 Major IT Project reporting templates and timelines & standard

QA contractor statement of work http://www.oregon.gov/DAS/EISPD/ITIP/IT_Investment_Oversight.shtml

38

Resources

IT Planning http://www.oregon.gov/DAS/EISPD/ITIP/pln_index.shtml

IT Oversight http://www.oregon.gov/DAS/EISPD/ITIP/IT_Investment_Oversight.shtml

IT Budget Development http://www.oregon.gov/DAS/EISPD/ITIP/IT_Budget.shtml

IT Lifecycle Planning http://www.oregon.gov/DAS/EISPD/ITIP/IT_Lifecycle_Planning.shtml

Business Case Development http://www.oregon.gov/DAS/EISPD/Business_Case.shtml

39

Resources

Project Management Institute (PMI - PMBOK) http://www.pmi.org/AboutUs/Pages/Standards.aspx

IT Infrastructure Library (ITIL)

ITIL V3 - http://www.itil-officialsite.com/home/home.asp International Standards Organization (ISO) 27001 & 27002

The standard is available to Oregon state employees by accessing the state of Oregon intranet at https://intranet.egov.oregon.gov/sites/DAS/EISPD/ESO/ISO.jsp

Information Systems Audit and Control Association (ISACA) COBIT V4.1 - http://www.isaca.org/

40

Contacts

Sean McSpaden, Deputy State CIO Phone: 503-378-5257 Cell: 503-798-1507 Email: [email protected]

Documents

Business Case Development and IT Project Oversight in the Government Environment NASACT Middle Management Conference April 13, 2011 Presented by: Sean