NET2866BU Learn from Challenging But Successful … · Network NSX for vSphere ... – Initially Detailed Runbook with Scripted Automated ESX host checks ... Critical Customers Partnering

Jeff Foraker : Johnson & JohnsonIT IS Senior Manager, Global SDDC DevOps, Network Virtualization

Chirag Patel : VMWarePrincipal Architect

NET2866BU

#VMworld #NET2866BU

Learn from Challenging But Successful NSX Deployment Journey with VMware SDDC at Large Pharmaceutical Company

VMworld 2017 Content: Not fo

r publication or distri

bution

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

CONFIDENTIAL 2



bution

Jeff Foraker : Johnson & JohnsonIT IS Senior Manager, Global SDDC DevOps, Network Virtualization

Chirag Patel : VMWarePrincipal Architect

NET2866BU

Learn from challenging but successful NSX Deployment Journey with VMware SDDC at large Pharmaceutical Company



bution

Session Abstract

• Over past three years, Johnson & Johnson and VMware collaborated on the planning, design, deployment, and operationalization of a Software-Defined Data Center across the globe. The solution offers fully-automated virtual machine provisioning, using a wide range of technology and software including vRealize Suite (vRealize Automation, vRealize Orchestrator, vRealizeOperation Manager, vRealize LogInsight), VMware NSX, Flash Storage and High-Performance computing platforms. The SDDC environment is the foundation of a modernization strategy based on simplifying and automating server, storage, and networking infrastructure using software-defined technology to enable refresh initiatives without major downtime and eliminate “technology-debt”.

• This Large Pharmaceutical is deploying SDDC for their main datacenters, remote offices and DMZ environment. This session will cover technical deployment details, best practices and lessons learned from this implementation.

4#NET2866BU CONFIDENTIAL



bution

SDDC Current State – 2017

Business agility • Reduced to Hours for provision.

• Creation of “As a Service” model for demand agility.

• Easily Pilot new ideas.

• Add Resources when needed, scale when

required.

• Break the hold of Physical hardware restrictions.

Availability • DR services built into SDDC POD.

• Workload Resiliency included in SDDC

Architecture.

• No longer required to manage availability

at the physical layer.

Scalability• Rapid Scaling of resource

• No long require a PO to add Hardware.

Financials • Usage model for consumption

• Remove HW TLM as a business requirement

• Scalable SDDC POD deployments are scalable.

• Faster Time-to-Market

• Business can fail fast, iterate,

learn – Agile/DevOps

• Adapt to market disruptions

• Get Ahead of disruptions

• Deliver On-Premise Cloud

• Improved Reliability & Security

• Built-in workload management

and security dramatically

reducing alerts

• Built in diagnostics and

escalation

• Enhanced change isolation

• Reduced time to resolve failures

Business Benefits

• Weeks to provision infrastructure

• Multiple touch points and forms

to provision

• Costly infrastructure to operate

• Opaque usage and cost allocations

• Complex and slow to scale

• Limited high-availability

and disaster recovery

2014 – Enterprise

Business/Customer Outcomes: “Driving from Legacy to True SDDC Benefits”

5



bution

StrategyPlanningRoadmap

IT Transformation

Global SDDC Rollout

Prove transformative power of SDDC Solutions

SDDC Strategy

• Viability of SDDC

• SDDC Business Case

• Establish SDDC Strategy

• Prove out SDDC “art of

the possible”

• Large ERP on SDDC PoC

• Target first major release

Initial SDDC DevOps Approach

• Built in “Fail fast” into DevOps Team

• Established SDDC Devops Leadership

• Built Executive transformation

sponsorship

• Partner with VMware Leadership

• Partnered with IT Leadership

• Created Dev/QA env for DevOps team

• Production SDDC rollout in Singapore

• Stress test of the SDDC environment

• Set Operational Goals and Business

Intent

• Establish Key milestones and Metrics

• Communicate to all Business units

and Customers

Global SDDC Implementation

• Agile, Agile, Agile

• Global deployment - 6 Sites WW

• Repeat Communication plan to

Customers

• VMware partnership and full

engagement

• Build out SDDC DevOps team and

support model

• Established customer DRI for workload

migrations

• 2015 : First DR/Colo Site in Malaysia &

First ERP app on SDDC

• Automation & Integration implementation

• Backup & Restore of entire SDDC stack

• Config./Design & Operations

Assessment

Q2 2014

2014

2015 - 2017

2017 - 2018

Global SDDC Expansion & Remote Sites

Deployment

• Automation & Integration Enhancements

(OS, DBaaS (SQL - Oracle), SAP BP)

• As of Aug 2017, ~ 18,678 VMs in SDDC

• Aggressive goals to migrate workloads

from Legacy

• Increased adoption of ERP applications

• Expansion of capabilities into SDDC DMZ

workloads

• Begin Rolling out SDDC at Remote Sites

• Global migrations complete to SDDC target

platform

• Achieve 40% Enterprise Application

Rationalization

SDDC Journey & Project Timeline

6



bution

SDDC Environment with VMWare vRealize Suite

▪ vRO for cloud orchestration

▪ vRA for policy based governance and service delivery

Network ▪ NSX for vSphere

Security ▪ NSX for vSphere

Hypervisor ▪ ESX / vSphere

Storage ▪ API-Based Storage Virtualization

LAN▪ Spine, Border Leaf layout

Storage▪ Tiered Storage Arrays

OS layer

& above

▪ MS Windows Operating System

▪ Linux operating systems

Servers

Private cloud

services

Backup

▪ X86 2-socket CPU

▪ Enterprise Backup solution based on Virtualization

Management tools

▪ vRA Configuration and compliance management through workload automation

▪ vROps for unified performance, incident and capacity management

▪ Integration of vROps with IT Service Management tools (CMDB and Event Mangement platforms.)

▪ Integration with Identity and Access Management

▪ Financial management and cost transparency

▪ Enterprise Backup solutions utilizing vDP for data backup

▪ Disaster Recovery through SRM

▪ NSX for Software Defined Networking

Vir

tua

liza

tio

nP

hys

ica

l L

aye

r

SD

DC

sta

ck

▪ Establish Release Management standards and DevOps approach

▪ Separate Hardware and Software for agility and break legacy mindset

▪ Standardized Maintenance Windows for SDDC platform

▪ Increase level of availability and up-time

▪ Improve Storage and Backup OLA rates

Design Criteria

SDDC Technology Stack

7



bution

Networking Architecture Overview

• Spine Leaf Physical Architecture in EDC

• ROBO sites have ICE environment with TORs

• 5.5 Everything is in single DLR

• 6.0 has DEV/QA/Prod DLR and backup DLR

• Initial deployment was single Active VTEP with standby. Since then we have multiple active VTEPs based on LB SRC ID

• HA Pair of ESG and now using ECMP

8



bution

Networking Architecture EDC

• Spine Leaf Physical Architecture in EDC

• Multiple compute PODs connected to spine

9



bution

Networking Architecture ROBO

• ROBO sites have ICE environment with TORs

• Segregation of levels as per ISA-95 standard

– Level 4 = Office network

– Level 3 = ICZ (Isolated computing environment DMZ)

– Level 2 = Automation networks

10



bution

Networking Architecture Virtual Network



11



bution




12



bution




13



bution

Capabilities & Features In Use

– Workload Mobility - VXLAN

• Within DC

– NSX LB for one of most important application in the environment - Application X

– Workload mobility using WAN and Local NSX components for Application X

• RTO = 15 mins, RPO = 0

– In Physical DMZ, using DFW for app isolation

– EDC Turned off DFW due to legacy application disconnect issues

14



bution

Capabilities & Features In Use – Application X Operational


15



bution

Capabilities & Features In Use – Application X Partial Failover


16



bution

Capabilities & Features In Use – Application X Full Failover


17

Add - NSX & SRM scripts used



bution

Architecture and Scaling Considerations

• Challenges and tweaking we had to do to

– ARP Default to 5,000 in earlier version

– Initial VTEP Pool (5.5 - /25) Same L2 Segment. New VTEP Pool /22

– NETCPA prior to 6.2.4

18



bution

Operational Considerations

• Upgrade

– VUM and EAM not working together

– Initially Detailed Runbook with Scripted Automated ESX host checks

– Tracking / Checklist

• Maintenance - Scripts, APIs

• DLR and ESG deployment has been automated

• Automated NSX Edge for LB

• Organization Challenges

– How traditional VI ops guy handle maintenance and access ESXi networking

• How to deal with separate physical networking team

19



bution

Operational Considerations - Upgrade

20



bution

Operational Considerations - Scripts

21



bution

Future Goals

• Future use cases

– WL Micro segmentation

– VMWare Mgmt stack on VXLAN and using LB to align with VVD

– vRA integration for additional capacity

• Add IP space from Infoblox

• Add that to DLR or add new DLR

– vRA integration for LB

– ESG FW in ROBO

– Workload Mobility Across DC & Across Cloud (Hybrid Cloud)

– NSX-T

22



bution

VMware and J&J Collaboration

• Improved Test Coverage

• Reboot less upgrade

– Host & Network Health Checks before host is active again

• Improved Third-party HW Vendor Driver Compatibility

23



bution

Lessons Learned

• Our Lessons learned

– Deploy Universal objects day 1

• HW Vendors driver release and compatibility coordination with VMWare

• E2E Jumbo frames (Including Physical Networking to achieve cross DC WL migration)

• Ownership of deployment with supporting expertise

• End to End understanding of virtual and physical stack

– Networking is Networking!

24



bution

Highly Suggested Session

• PBO2794BU : Data Archiving for VMware SDDC Using NetBackup: Learn from This Large Successful VMware SDDC Deployment Journey at a Large Pharmaceutical Company

• NET2866BU : Learn from challenging but successful NSX Deployment Journey with VMware SDDC at large Pharmaceutical Company

• MGT2898PU : Pushing the Limits: Critical Customers Partnering with VMware Engineering

• DEV1519PU : DevOps in the Real World: Customer Panel

• NET1777BU : Troubleshooting Methodology for VMware NSX for vSphere

25



bution



bution



bution

Documents

NET2866BU Learn from Challenging But Successful … · Network NSX for vSphere ... – Initially Detailed Runbook with Scripted Automated ESX host checks ... Critical Customers Partnering