SCUG.DK - Automation Strategy - April 2015

Automation Strategy

Greg_Ramsey@dell.com | @ramseyg | www.ramseyg.com

Automation – two (primary) types at Dell

• Workload Automation – Batch processing, rich scheduling (BMC Control-M) – 15K jobs per day

• Orchestration/Automation Workflows – Tying tools together. System Center Orchestrator.

Why Automate?

• No one receives job satisfaction from pushing the same button every day (redundant tasks). Give our team more exciting tasks.

• We will never have more staff to maintain our environment than we do today, regardless of company performance.

• If three years from now, we are still manually doing the same operational tasks as we are today, we have failed.

How automation can help you

Work order automation

Access management automation

Incident automation

Change management automation

Provisioning automation

Solution enabler for your team

Manual process automation

Automation operation support

Data integration automation

Jobs scheduling automation

Automation consultant and

coaching

Yesterday….

L3

L2

L1

• Work Escalated INC, work with L2 to create KB• Design Infrastructure improvements• Collaborate with other teams

• Work escalated INC, create KB• Identify trends and escalate.• Implement CRQ• Work SRM

• Work INC, according to KB• Implement CRQ• Work SRM

• Incidents• SRM• CRQ

Desired State - Automation of Operational Tasks

L3

L2

L1

VirtualEngineer

• Gather trends from L2 and L1, and work to automate. (Based on KBs)

• Design Infrastructure improvements• Collaborate with other teams

• Handle incident exceptions, create new KBs• Review problem management and/or top SRM Requests,

partner with L3 to automate

• Handle exceptions that are not automated• Identify trends and escalate.

• Perform automated INC/SRM request automation

• Incidents• SRM• CRQ

Mo

ve the o

peratio

nal w

ork

Automate the operational work

Move operational tasks toward Virtual Engineer

• The more tasks we can transition to Virtual Engineer, the more technical tasks can be moved toward L2 and L1.

• Labor cost savings by moving more tasks from L3->L2 and L2->L1

• Faster Resolution, Improved End-User Experience

Engagement Process

Engagement Options

Partnership

• You prioritize

• You do 90% of the work

• We provide governance, L3 Consultation

• We release to production

Full Service

• Our Business Analyst

• Our team Builds Automation Integration Packs

• *manage shared runbooks

• Custom Build

• Agile Process, with Prioritization

Engagement process details

Intake Portal (SharePoint)

TFS - Story

Standards

• Runbook Standards

– Admin Accounts

– Ports

– Error Handling and logging

• Integration Standards

– ITSM (SRM, Change and INC)

– Custom integration packs

• Reporting Standards

• Monitoring Standards

Standards – Automated Incident Resolution

Automated Incident Resolution Standards

Measuring Automation

Measuring Automation – SRM 1

SRM Request Created

Request Assigned

24 h 48 h 72 h 96 h IT SLA

IT Work Start

IT Work Complete

1 hr

SRM Complete

64 h End User Experience Time

1 hr IT Time

Measuring Automation - SRM 2

SRM Request Created

Request Assigned

3 min 4 min 5 min 6 min IT SLA

Automation Start

Automation Complete

SRM Complete

6 min End User Experience Time

1.3 min IT Time

IT Improvement: 58.7 min (99.978%)End User Improvement: 3834 min (99.998%)

Measuring Automation - IPMS– Audit Script

IPMS Notification

12 h 24 hr 36 h 48 hIT SLA

Request Assigned

Audit Start Audit Compete

40 h process time

3 h IT Time

IPMS Workflow

2 days

Request Completed

Measuring Automation - IPMS– Audit Script

IPMS Notification

3 min 4 min 5 min 6 minIT SLA

Request Assigned

Audit StartAudit

Compete

5.5 min process time

2 min IT Time

2 days

Request Completed

IT Improvement: 178 min (99.989%)End User Improvement: 2394.5 min (99.997%)

Crunching the Numbers…

IT Time

Process Time

Before (min)

180

2400

After (min)

2

5.5

Percent Improvement

99.989

99.998

Time Reduction

178

2394.5

Iterations Per Year

1000

1000

Total Savings (min)

178000

2394500

Total Savings (hour)

2,966.67

39,908.33

Employee Hours/year 2080

Cost of L1 Employee/year $70,000

Potential Employee Savings/year 1+

Bonus!

Crunching the Numbers…BONUS!

IT Time

Process Time

Before (min)

180

2400

After (min)

2

5.5

Percent Improvement

99.989

99.998

Time Reduction

178

2394.5

Iterations Per Year

5000

5000

Total Savings (min)

890,000

11,972,500

Total Savings (hour)

14,833

199,541

Employee Hours/year 2080

Cost of L1 Employee/year $70,000

Potential Employee Savings/year 7+

Measuring

Type Value

IT Time Savings

User Experience Improvement

Value-Add (stuff we never had time to do, but can now do because of automation)

R

Lifecycle

• Treat a runbook as an application in your environment.

– Process owner

– Quarterly Reviews

– Measure Value

– Monitor and react to runbook failure

Questions

Greg_Ramsey@dell.com | @ramseyg | www.ramseyg.com

Some examples