IT Operations @ Microsoft

IT Operations @ MicrosoftIT Operations @ Microsoft

André HowardSenior Operations Analyst - UK

IntroductionIntroduction

Three things to take away todayThree things to take away today

Complex but works joined upComplex but works joined up Less is often moreLess is often more We don’t have a monopoly on best practiceWe don’t have a monopoly on best practice

AgendaAgenda


•StrategyStrategy

•StandardisationStandardisation

•CentralisationCentralisation

•AutomationAutomation

Operations StrategyOperations Strategy


SydneySydney

Chofu & OtemachiChofu & Otemachi

Les UlisLes UlisTVPTVP

DublinDublin BeneluxBenelux

MadridMadrid

DubaiDubai

SingaporeSingapore

JohannesburgJohannesburg

Sao PauloSao Paulo

72,000 72,000 mailboxesmailboxes

Microsoft’s Worldwide ITMicrosoft’s Worldwide IT

Canyon Park,Canyon Park,RedmondRedmond

Los ColinasLos ColinasCharlotteCharlotte

ChicagoChicagoMilanMilan

StockholmStockholm

MunichMunich

400+ 400+ supportesupported MS d MS sites sites worldwidworldwidee

4.5M+ e-mail 4.5M+ e-mail messages per day messages per day internallyinternally

>26M NT >26M NT eventsevents>400 apps>400 apps26M voice calls 26M voice calls per monthper month50K employees50K employees5K contractors5K contractors17K vendors17K vendors

150,000+ PCs150,000+ PCs>7,000 >7,000 serversservers

110 110 Exchange Exchange Messaging Messaging ServersServers

Silicon ValleySilicon Valley

Infrastructure OperationsInfrastructure Operations

AccountManagement

AccountManagement

SiteOperations

SiteOperations DataCenterDataCenter MessagingMessaging NetworkNetwork

Tier 3Tier 3

EngineeringEngineering ServiceManagement

ServiceManagement

Regional Functions Central Functions

Operational Area

Strategic Area

HelpdeskHelpdesk

ProgramManagement

ProgramManagement

Operations: PastOperations: Past

NetworkNetwork Poor Technology/Low MTBFPoor Technology/Low MTBF Low Bandwidth & High CostLow Bandwidth & High Cost Poor ScalabilityPoor Scalability

HardwareHardware No Standards for systemsNo Standards for systems Poor Monitoring of services/systemsPoor Monitoring of services/systems High Costs for maintenance/purchaseHigh Costs for maintenance/purchase

Distributed EnvironmentDistributed Environment Duplication of Systems/Applications/DataDuplication of Systems/Applications/Data High Costs for management/supportHigh Costs for management/support

EDC

RDC

Operations: PresentOperations: Present

Enterprise Data CenterEnterprise Data Center Origin point for NetworkOrigin point for Network Tier 3 staff, 7x24 monitoring, RDC/SDC Tier 3 staff, 7x24 monitoring, RDC/SDC

remote monitoring, host application remote monitoring, host application servers regional and global, RDC/SDC servers regional and global, RDC/SDC escalation pointescalation point

Regional Data CenterRegional Data Center Tier 2 staff, 7x24 support, Redundant Tier 2 staff, 7x24 support, Redundant

Network Backbone connection, Hub Site Network Backbone connection, Hub Site for SDCfor SDC

Host applications, Exchange, Public Host applications, Exchange, Public Folders, Internet Proxy, Mail Gateway, Folders, Internet Proxy, Mail Gateway, RAS, Backup/Restore, DDS Server, RAS, Backup/Restore, DDS Server, DHCPDHCP

Site Data CenterSite Data Center Network connection to RDCNetwork connection to RDC Login service, File Server with local Login service, File Server with local

share points, Print Servershare points, Print Server

SDCS

ervices & A

pp

lication

s

A Strategy for ChangeA Strategy for ChangeGoals:Goals: Improve Efficiency Improve Efficiency

Be SmarterBe Smarter

Reduce Time to Market Reduce Time to Market Improve StandardsImprove Standards

Reduce Costs/Headcount Reduce Costs/Headcount We do not have access to a never ending pot of moneyWe do not have access to a never ending pot of money

Increase Availability/Quality Increase Availability/Quality but do not reduce servicebut do not reduce service

Methods:Methods: StandardisationStandardisation Centralisation (Remote Operations)Centralisation (Remote Operations) AutomationAutomation Lights OutLights Out Data Centre TaxonomyData Centre Taxonomy

IT ServiceAreas

PartnerSite

ConnectionType

ConnectionState

Network RedundancyConnection

Monitored by GOTIT Equip Monitored

By GOTCE Archetype

FacilityLabor

MS Site Types

Multiple Sites No Direct Persistent

High Availability Buildings must have

diverse access (2 paths) Yes Yes (N+1)*2 24x7 Data Center 4



diverse access (2 paths) Yes Yes 2N 24x7 Data Center 3



diverse access (2 paths) Yes Yes (N+1)

Max: 24x7Min: On Call Data Center 2



diverse access (2 paths) Yes Yes N

Max: 8x5Min: On Call Data Center 1

Single local site No Direct Persistent


diverse access (2 paths) Yes Yes N

Max: 8x5Min: On Call

High Availability Building (HA)

Single local site No Direct

VOIP toll bypass is part of the costing analysis for determining savings

for ICO-1 and ICO-2 None Yes Yes No Back-up CE systems Std Commercial Standard Building

Single local site No Internet Persistent None Yes Yes No Back-up CE systems Std CommercialInternet Connected Office (ICO) - 2

Single local site No Internet Non-Persistent None No No No Back-up CE systems Std CommercialInternet Connected Office (ICO) - 1

Single local site Yes Direct Persistent None Yes No No Back-up CE systems Std Commercial Partner-2Single local site Yes Internet Non-Persistent None No No No Back-up CE systems Std Commercial Partner -1

Operations: Future – A Dream?Operations: Future – A Dream?Model Enterprise Infrastructure ~ MEIModel Enterprise Infrastructure ~ MEI

Data Centre TaxonomyData Centre Taxonomy

Operations: Future – A Dream?Operations: Future – A Dream?Model Enterprise Infrastructure ~ MEIModel Enterprise Infrastructure ~ MEI

Data Center 4Data Center 4 Origin point for NetworkOrigin point for Network Tier 3 staff, 7x24 monitoring, Tier 3 staff, 7x24 monitoring,

RDC remote monitoring, host RDC remote monitoring, host application servers regional application servers regional and global, Exchange, Public and global, Exchange, Public Folders, DHCP, Login service, Folders, DHCP, Login service, SAN’s for File Storage, RDC SAN’s for File Storage, RDC escalation pointescalation point

Data Centre 2Data Centre 2 Tier 2 staff, 7x24 support, Tier 2 staff, 7x24 support,

Redundant Network Backbone Redundant Network Backbone connectionconnection

Internet Proxy, Mail Gateway, Internet Proxy, Mail Gateway, RAS, Local Backup/Restore, RAS, Local Backup/Restore, DDS Server, Print ServerDDS Server, Print Server

DC4

DC2

Operations StandardizationOperations Standardization


Server Standards - HardwareServer Standards - Hardware

Hardware Engineering Provides tested and application engineered systems Build documentation for hardware Escalation Support

Server Standards - SoftwareServer Standards - Software

OTG Service Packs (IPAK) Installation of all core OS and applications i.e. SQL,

IIS via scripts Build documentation for software Escalation Support

Change ManagementChange Management

OTG Change Management Enables tracking of new builds, procurement, moves,

adds and changes Single Web interface supports our global sites Built in approval procedure

Change ControlChange Control

OTG Change ControlOTG Change Control Communication to our businessCommunication to our business Notification to our peersNotification to our peers Search to see what’s comingSearch to see what’s coming

Operations CentralisationOperations Centralisation


Remote Server ManagementRemote Server Management Terminal Services:Terminal Services:

Fast & easy remote Server Fast & easy remote Server administrationadministration

Remote Insight Board (LOE):Remote Insight Board (LOE): Server independent Server independent

separate Power Supply, GUI, separate Power Supply, GUI, Keyboard/Mouse, NIC, integrated Web Keyboard/Mouse, NIC, integrated Web serverserver

Full remote accessFull remote access independent from OS or system state of independent from OS or system state of

the serverthe server Remote ConsoleRemote Console Remote Floppy driveRemote Floppy drive Remote power on/off of the serverRemote power on/off of the server

Server Monitoring - ConceptServer Monitoring - Concept

CIM EventsCIM Events

Dell OM EventsDell OM Events

BackOffice EventsBackOffice Events

NetIQ EventsNetIQ Events

Application EventsApplication Events

Helios EventsHelios Events

Httpmon EventsHttpmon Events

HPOV EventsHPOV Events

OS EventsOS Events

TicketSystem

TicketSystem

MOMEvent DB

MOMEvent DB

GNS AssetECM

Config DB

GNS AssetECM

Config DB

MOMServer Event

Yellow Console

Polling EventsPolling Events

3rd Party Events3rd Party Events

ECM EventsECM Events

Challenges of CentralisationChallenges of Centralisation

Multiple Languages Multiple Currencies Different working weeks Different time zones

Operations AutomationOperations Automation


Automated Share AdministrationAutomated Share Administration

Share Administration Enables clients to self serve share access Single interface supports our global sites Built in approval procedure

Automated DL/SG AdministrationAutomated DL/SG Administration

Distribution List & Security Group Administration Enables clients to self serve DL & SG access Single interface supports our global sites Built in approval procedure

Automation – for youAutomation – for you

What do you do that is repetitive?What do you do that is repetitive? What can you automate?What can you automate? How much money would you save?How much money would you save? How much time would you save?How much time would you save?

In closing...In closing...

Complex but works joined upComplex but works joined up Less is often moreLess is often more We don’t have a monopoly on best practiceWe don’t have a monopoly on best practice

You are IT, We are IT, lets work togetherYou are IT, We are IT, lets work together

Documents

IT Operations @ Microsoft