Optimizing Oracle Exadata with Oracle Support Services [CON7054]

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Optimizing Oracle Exadata with Oracle Support Services [CON7054]A Client View from KPN

Eric ZonneveldArchitect Engineered SystemsOracle Advanced Customer Support

Jan van DijkenService Delivery EngineerOracle Advanced Customer SupportOctober 02, 2014


Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program Agenda

KPN Overview

Platinum Services

Advanced Monitoring and Resolution

Advanced Customer Support

Lessons Learned

1

2

3

4

5


“We have multiple suppliers who deliver support and updates, but that makes it difficult for us to meet our service levels. We believe Oracle Platinum Services can improve the way we support our environment so that we are able to meet our service levels and, at the same time, lower our costs.”

Michel BosmanLead Architect, KPN

Play video

http://medianetwork.oracle.com/video/player/1774105164001

http://medianetwork.oracle.com/video/player/1774105164001


KPN OverviewInfrastructure, project and organization


KPN Overview

2010-2011• Project startup (Mar)

• Ordering (Nov)

• POC Systems delivered X2-2 quarter

(Dec)

• Phase I: Initiation and testing (Jan)

• Phase II: POC and pilots (Feb)

• Phase III: EXA demo (Mar)

• Phase IV: DTAP systems delivered X2-2

full (Jun)

• First critical application chain live (Nov)

2012-2013• Platinum Services (Aug)

• DTA systems delivered X3-8 full (Jan)

• Oracle ACS and AM&R contract for

service delivery (IM, PM, CM) (Jan)

• EXATEAM created (Jan)

• Migration DTA -> X3 (Jun)

Separate production and non-

production at rack level

2014• Delivery second production P2 based

on 12c (Jan)

• Migrate workload P1 to P2 (Apr)

• Exalogic (May)

• Exalytic (Aug)

Project


Haarlem

Dataguard

40 km

Aalsmeer

KPN OverviewInfrastructure


KPN OverviewTotal Hardware

• 24 compute nodes

• 62 storage nodes

• 560 cores

• 10,8 TB memory

• 55,8 TB flash

• 1,824 TB storage

• 16 IB switches• 6 CISCO switches

• 4 KVM

Per Site

• Storage bandwidth 50GB/s X293GB/s X3 per rack

• 1,5M IOPS (read) 1M IOPS (write) per full rack


KPN Overview

• Interesting landscape …– Huge Oracle database landscape (>2,000), lots of Oracle middleware and applications

– All hardware and OS platforms in 27 datacenters

– All versions (e.g. Oracle database 7 and up)

• Interesting objectives …– Data center migration and application / chain consolidation

– Company strategy is to have vendor / partner take more responsibility

– Decrease spend

Challenges


KPN Overview

• Which leads to …– Increasing workloads because of consolidation

– Increasing demands for performance, stability and availability

• While at the same time …– Aggressive outsource strategy

– Continuous re-organization

• Challenging indeed

Challenges


KPN Overview

• “Old fashioned” support did not work– Tiered model

–Offshore

• “New way” of working– DBA centric, end-2-end responsible

– 1 team onshore

– Strong combination between Oracle (run) and KPN (change)

How to engage


KPN Overview

• Oracle Platinum Services– Change management (Patching)

• Oracle Advanced Support Cloud Services– Incident management, problem management

• Oracle Onsite Support– Innovation, life cycle management (infra upgrade)

• KPN ITS Platform Business Applications– Change management, business consulting, backup & recovery

Organization


KPN Overview

• Scalability– Servicing 594 databases (incl. 132 standby databases)

– 220TB storage used (net)

– 58 applications

• Performance– Performance gains ranging from 2x to 100x

– Special hardware and backup infra no longer needed

– Application scaling up 10x volume without a problem

Benefits of introducing Exadata


KPN Overview

• Availability– Availability increase (no unplanned downtime)

– Real Application Clusters

– Active dataguard

–Maximum availability architecture

• Faster life cycle management–Most up-to-date and secure platform

– Introduction of 12c at least a year before other platforms



KPN Overview

• But what about support?– Incident Management SLA’s (notification, diagnostic, resolution)

–MTTR

– Service request handling

– Problem management

– Business KPI’s (anything)



Platinum ServicesMake the invisible visible


Platinum Services

• Part of hardware support

• “Free” but with rules

• Limited scope for incident management

• Oracle Advanced Support Gateway server

• OEM12c, customized with metrics, thresholds and escalation rules

• Patching

• Change management (in ACS portal)

What is it?


Platinum Services

• Visibility of incidents (not events!)

• Visibility of problems

• Visibility of (wrong) configuration– RDS in Oracle executable

• Visibility of Oracle best practices not being applied

• Huge backlog–Moved from >120 open SR’s after engagement to average 15 SR’s currently

What changed?


Oracle MonitoringAdvanced Support Gateway

• Platinum aids companyDBA’s with the incident management process


Oracle Advanced Support Cloud ServicesAdvanced Monitoring and Resolution


Oracle Advanced Support Cloud Services

• Extends Platinum Services, adds problem management

• Broader scope– More events

– Diagnostics

– RCA’s

– Resolution

• Integration with client IM/PM processes (ITIL-based)– Service Now

What is it?



• More responsibility delegated to Oracle (IM and PM process)

• Shifts focus from KPN DBA’s from running the infrastructure (run) to aid applications and business processes, introduce new functionality (change)

• Advanced Support portal

What changed?



• Increased service level objectives, e.g.– Resolution• Severity 1: 80% < 1 day, MTTR average < 1 day

• Severity 2: 80% < 3 days, MTTR average < 3 days

– Availability reporting• Of (by the client) nominated critical database services and related applications, >99.99%

– Incident summary reporting, quarterly• Production / non-Production, by date and severity, by category (Exadata software, database,

compute node, storage, switch, other)

What changed?


Oracle Onsite SupportCustomer Integration


Oracle Onsite Support

• Acts as the “glue” between Oracle Platinum, AM&R and the client

• Life cycle management (upgrades)

• Innovation– New functionality, new releases

– Best practices, Standardization

• Expert troubleshooting

• On site, close contact, customer intimacy

What is it?


Lessons Learned2 years of engagement, what worked and what didn’t?


Lessons Learned

• Oracle should be more visible in what they do– Frequent face-to-face reporting in a format that appeals to the customer

• Client does not want to know the differences within Oracle (Platinum, AM&R and ACS), should be one team and one voice

• Incident to problem promotion can be improved

• Portal could be improved– Google search and user friendliness

– CI tagging

Room for improvement


Lessons Learned

• Continuous learning within AM&R / Platinum– Enhancement of KA’s (knowledge articles) must become second nature

• Change management– Integration of Oracle changes into the CM system and processes of the client

–Work on “awareness” of customers of patch frequency, hardware maintenance etc.

• Communication between support engineers and customer– Do not assume customers read all mail or plough through the portal every day

Room for improvement


Lessons Learned

• Insight into Exadata significantly improved (exposure of problems)

• Co-operation between KPN and Oracle– Service delivery management (TAM)

• Co-engineering between Oracle and KPN– Local ACS resources

– Complex problem solving

What worked?


Oracle ServicesAdvanced Customer Support

+ + =

and Support

Documents

Optimizing Oracle Exadata with Oracle Support Services [CON7054]