EXL311: Exchange Server 2013 Architecture Deep DiveScott SchnollMicrosoft Corporationscott.schnoll@microsoft.comhttp://aka.ms/schnoll @schnoll

EXL311

About this Presentation

Capabilities are subject to change

Packaging and licensing have not yet been determined

Any screen captures or concepts shown are pre-release and for illustration purposes only

DisclaimerThis presentation contains preliminary information that may be changed substantially prior to final commercial release of the software described herein. The information contained in this presentation represents the current view of Microsoft Corporation on the issues discussed as of the date of the presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of the presentation. This presentation is for informational purposes only.

MICROSOFT MAKES NO WARRANTIES, EXPRESSED, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this presentation. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this information does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

All performance statistics are pre-release and subject to change

Agenda Architectural Changes

Client Access server role

Mailbox server role

Exchange Server: The Evolution

L7 LB

2010

• Separate HA solutions for each role

• Introduced the DAG• Rich management

experience using RBAC

• Support for Hybrid deployments

CAS

HT

MBX MBX

2007

• Separate roles for ease of deployment and management segmentation

• Support cheaper storage

Ex Ex

SAN

Ex Ex

2000/2003

• Role differentiation through manual configuration

• Backups and hardware solutions for “reliability”

?

Previous Server Role Architecture

5 server roles

Tightly-coupledin terms of

versioningfunctionalityuser partitioninggeo-affinity

Internal Network Phone system (PBX or VOIP)

Web browser

Outlook (remote user)

Mobile phone

Line of business application

MailboxStores mailbox

and public folder items

Unified MessagingVoice mail and

voice access

Client AccessClient

connectivityWeb services

Outlook (local user)

Layer 7 LB

AD

ExternalSMTP

servers

Edge TransportRouting and

AV/AS

Hub TransportRouting and

policy

Forefront Online Protection for

Exchange

Challenges with existing model

Exchange deployments can be complicated

Load balancing is difficult and can require expensive solutions

When dedicated server roles are deployed, hardware can go unutilized or under-utilized

Too many namespaces required

Architecture Changes

Exchange 2013 Architecture Theme

Use Building Blocks to facilitate deployments at all scales – from self-hosted, small organizations to Office 365• Server role evolution• Network layer

improvements• Versioning and inter-op

principles

Exchange 2013 Architecture Benefits

Hardware efficiencyDeployment simplicityCross-version inter-opFailure isolation

Exchange 2013 Server Role Architecture

Building BlocksClient Access server• CAS ArrayMailbox server• DAG

Loosely-coupledFunctionalityVersioningUser partitioningGeo-affinity

Internal Network

ExternalSMTP

servers

Phone system (PBX

or VOIP)

Web browser

Outlook (remote user)

Mobiledevice

Line of business

application

AD

DAG

Layer

4 L

B

CAS Array

Exchange Online Protection

Outlook(local user)

Exchange 2010Edge Transport

Protocols, Server Agents

Business Logic

Storage

EWS

RPC CA

Transport

Assistants

MRSMRSProx

yEWS

RPC CA

Transport

Assistants

MRSMRSPro

xy

Server1 (Vn) Server2 (Vn+1)

XSO MailItem

Other API

CTS

Store

ESE

Contentindex

File system

XSO MailItem

Other API

CTS

Store

ESE

Contentindex

File system

SMTP

MRS proxyprotocol

EWS protocol

Custom WS

Banned

Exchange 2013 Tenet: Every Server is an Island

E2010

Functional Layering

AuthN, Proxy, Re-direct

Protocols, API, Biz-logic

Assistants, Store, CI

Exchange 2010Architecture

AuthN, Proxy, Re-direct

Store, CI

Protocols, Assistants, API,

Biz-logic

Exchange 2013Architecture

Client Access

Mailbox

Client AccessHub Transport,

Unified Messaging

Mailbox

HardwareLoad Balancer

L4 LB

L7 LB

Client Access Server Role

Client Access Server role in Exchange Server 2013

Domain-joined machine in the internal Active Directory forest

Thin, stateless (protocol session) server

Comprised of three components:Client access protocols (HTTP, IMAP, POP)SMTPUM Call Router

Exchange-aware proxy serverUnderstands requests from different protocols (OWA, EWS, etc.)Supports proxy and redirection logic for client protocolsCapable of supporting legacy servers with redirect or proxy logicContains logic to route specific protocol requests to their destination end-point

Client Access Array in Exchange Server 2013

A group of CAS organized in a load-balanced configuration

Designed to work with TCP affinity (aka, layer 4 LB)Does not require session affinity (aka, layer 7 LB)

Provides a unified namespace and authentication

Similar to Exchange 2010 in terms of providing a unified endpoint for client connectivity and authentication

Load Balancer

MDB

HTTP Proxy

IISClient Acces

s

RPC CA

Mailbox

IIS

RPS OWA, EAS, EWS, ECP, OAB

POP, IMAP SMTP UM

POP IMAP

Transport UM

SMTPPOP, IMAPHTTP

MailQ

Client Protocol Architecture in Exchange 2013

RpcProxy

SMTP

SIP

Redirect

SIP + RTP

POP/IMAPOutlook Web App Outlook EAS EAC PowerShell

Outlook Connectivity in Exchange 2013

Exchange 2013 supports RPC/HTTP only; No RPC/TCP

Simplifies the protocol stackProvides an extremely reliable and stable connectivity model because RPC session is always on Mailbox server hosting active copyEliminates need for RPC CAS Array namespace(s)Eliminates end user interruptions like “The Exchange administrator has made a change that requires you to quit and restart Outlook” during mailbox moves or *overs

Namespace Simplification

Exchange 2013 no longer requires multiple namespaces for site resilient solutions or site specific scenariosEasy to setup a single, worldwide client access namespace

Can be used in coexistence with Exchange 2010

NA-DC2NA-DC1

mail.contoso.com

VIP #1 VIP #2

DAG

A Single Common Namespace ExampleRound-Robin DNS

DNS Resolution

Sue (Somewhere in

NA)

Round-Robin between # of

VIPs

APAC-DC2APAC-DC1NA-DC2NA-DC1

mail.contoso.com

VIP #1 VIP #3VIP #2 VIP #4

DAG DAG

A Single Common Namespace ExampleGeographical DNS Solution

DNS Resolution via Geo-DNS

Round-Robin between # of

VIPs

Sue (Somewhere in

NA)

Sue (Traveling in

APAC)

DNS Resolution via Geo-DNS

Round-Robin between # of

VIPs

Benefits of new architecture

Simplifies the network layer

Removes need for RPC CAS Array

Provides deployment flexibility

Front-End Transport Service

Front-End Transport Service

Handles all inbound and outbound external SMTP traffic for the organization

Does not queue mail locally and is stateless

Functions as a layer 7 proxy and has full access to protocol conversation

Listens on TCP25 and TCP587

Front-End Transport Service Architecture

Front-End Transport Pipeline

SMTP Receive SMTP Send

Protocol Agents

SMTP to MailboxSMTP from Mailbox

External SMTP External SMTP

Hub Selector

Delivery Groups

Active Directory site, Mailbox, DAGServer selection within the delivery group is based on recipient

If message only has a single mailbox recipient, select Mailbox server within delivery group based on proximity of AD siteIf multiple mailbox recipients, select Mailbox server in closest delivery group, factoring in site proximityIf there are no mailbox recipients, select a random Mailbox server giving preference to local AD site

Front-End Transport Service Benefits

Centralized, load-balanced point of egress/ingress

Connection, recipient, sender, and protocol filtering

Avoids unnecessary hops by determining the best Mailbox server to deliver the message

Scales based on number of connections – just add more servers

Mailbox Server Role

Mailbox Server Role

Server that hosts the components that process, render and store Exchange data

Includes components previously found in separate roles

Only Client Access servers connect directly to the Mailbox server

Clients connect to Client Access serversConnectivity to a mailbox is always provided by the protocol instance local to the active database copy

Database Availability Group

Collection of servers that form a unit of high availabilityBoundary for replication and *overDAG members can be in different sitesCan have a maximum of 16 Mailbox servers

MBX1

MBX2

MBX16

Mailbox-related changes

Managed Store

IOPS reductions

Larger mailbox support

Modern public folders

New search infrastructure

Managed Store

Managed Store

Store service process (Microsoft.Exchange.Store.Service.exe)

Manages worker process lifetime based on mount/dismountLogs failure item when store worker process problems detectedTerminates store worker process in response to “dirty” dismount during failover

Store worker process (Microsoft.Exchange.Store.Worker.exe)

One process per database, RPC endpoint instance is database GUIDResponsible for block-mode replication for passive databasesFast transition to active when mountedTransition from passive active increases ESE cache size 5X

Microsoft Exchange Replication service

MSExchangeRepl.exeDetecting unexpected database failuresIssues mount/dismount operations to StoreProvides administrative interface for management tasksInitiates failovers on failures reported by ESE, Store and Responders

ESE Cache Management

Algorithm allocates memory for ESE cache for store worker processes based on RAM (max cache target)ESE cache allocated to each database (store worker process) based on number of local database copies and value of MaximumActiveDatabases

Static amount of cache allocated to passive and active copies

Store worker process will only use max cache target when operating as active

Passive database allocates 20% of max cache targetMax cache target computed at service process startup

Restart service process when adding/removing copies or changing maximum active database configuration

IOPS Reductions

IOPS Reductions

Improvements to logical contiguity of store schemaProperty blobs are used to store actual message propertiesSeveral messages / page means fewer large IOs to retrieve message propertiesUse of long-value storage is reduced, though when accessed, large sequential IOs are used

Reduction in passive copy IO100MB checkpoint depth reduces write IOTransaction log code has been refactored for fast failover with deep checkpoint

IOPS Reductions

Exchange 2003 Exchange 2007 Exchange 2010 Exchange 20130

0.2

0.4

0.6

0.8

1

DB IOPS/Mailbox

IOPS/Mailbox

+97% Reduction!

Larger Mailboxes

Support for Larger Mailboxes

Large Mailbox Size is 100 GB+

Aggregate Mailbox = Primary Mailbox + Archive Mailbox + Recoverable Items1-2 years of mail (minimum)

Increase IW productivityEliminate or reduce PST filesEliminate or reduce third-party archive solutionsOST size control with Outlook 2013

Time Items Mailbox Size

1 Day 150 11 MB

1 Month 3300 242 MB

1 Year 39000 2.8 GB

2 Years 78000 5.6 GB

4 Years 156000 11.2 GB

Elimination of Scheduled Maintenance

Recurring maintenance implemented within time-based assistant (TBA) infrastructure as two assistants:

StoreMaintenance: lazy index maintenance, isintegStoreDirectoryServiceMaintenance: disconnected mailbox expiration

Workload Management monitors CPU, RPC Latency, and replication health

Task execution throttled/deferred when resource pressure exists

Background ESE database scanning further throttledBased on datacenter disk failure analysis, target to complete background database scan within 4 weeks (based on multiple databases on 8 TB disks)

Periodic tasks to generate mailbox quota notification removed

Quota notifications generated at logon time

Modern Public Folders

Modern Public Folders

Public folders based on the mailbox architecture Single-master model

Hierarchy is stored in a PF mailbox (one writeable)Content can be broken up and placed in multiple mailboxesThe hierarchy folder points to the target content mailbox

Because it’s a mailbox, it’s in a mailbox database…thus,

High availability achieved through continuous replicationNo separate replication mechanism

Similar administrative features to current PFs

No end-user changes

MBX2013

CAS2013

MBX2013

MBX2013

Public logon

Private logon

Public logon

Content MailboxHierarchy

Mailbox

Modern Public Folders1 - User connects to their home

Public Folder mailbox first, which should be located near their primary mailbox.

2- Folder contents live in one specific mailbox for that folder. All content operations are redirected to the mailbox for that folder

3 – Folder hierarchy changes are intercepted and written to writeable copy of Public Folder hierarchy

4 – All Public Folder mailboxes listen for hierarchy changes and update similar to Outlook clients

5 - When a Public Folder mailbox gets full, move some folders to a new mailbox

1

2 3 5

4

New Search Infrastructure

New Search Infrastructure

Uses FAST

Significantly improved query performance

Significantly improved indexing performance

FAST Primer

FAST Core

Catalog

CTS

Incoming Documents

FilterWord Break

Content

XForm

MARS Write

r

Incoming Queries

“CTS Flow”

IMSContent XForm

Query

Parse

“IMS Flow”

Res

ults

Mailbox

DB

Idx

Passive

Exchange Search Infrastructure

TransportTransport CTS

MailboxStore

DB

Index Node

Idx

ExSearch

Loca

l Del

iver

y

Reliable

Event

CTS

Read Content

MBX2013

LogLog

MBX2013

Transport-related Changes

Transport Components

Transport on Mailbox server is three servicesMicrosoft Exchange Transport - Stateful and handles SMTP mail flow for the organization and performs content inspectionMicrosoft Exchange Mailbox Transport Delivery - Receives mail from the Transport service and deliveries to the mailbox databaseMicrosoft Exchange Mailbox Transport Submission - Takes mail from the mailbox databases and submits to the Transport service

Transport has the following responsibilitiesReceives all inbound mail to the organizationSubmits all outbound mail from the organizationHandles all internal message processing such as transport rules, content filtering, and antivirusPerforms mail flow routingQueue messagesSupports SMTP extensibility

Transport Service Architecture

Transport Pipeline

SMTP to MBX Transport Submission

SMTP from MBX Transport Delivery

SMTP SMTP

Delivery Agents for other protocols

Submission Queue

Delivery Queue

Delivery Queue

Pickup/Replay

Categorizer

Routing Agents

SMTP Send

SMTP ReceiveProtocol Agents

Mailbox Transport SubmissionMailbox Transport Delivery

Mailbox Transport Component Architecture

Mailbox Transport Pipeline

Store Driver Deliver

MBX Deliver Agents

SMTP SendSMTP Receive

Hub Selector (Router)

Store Driver Submit

MBX Assistants

MBX Submit Agents

MAPI MAPI

Mailbox Store

SMTP to Transport Service

SMTP from Transport Service

Mailbox Transport Component

Two separate services to handle mail submissions (from the store) and mail delivery (from the Transport service)Mailbox Assistant and Store Driver combinedLeverages SMTP (encrypted) for communication with the Transport component and TCP465 for inbound trafficLeverages local RPC for delivery to storeIs stateless and does not have a persistent storage mechanism

Message Delivery

When receiving a message, the Mailbox Transport component can either deliver the message or not deliver the messageIf non-delivery is chosen, then the Mailbox Transport component must provide a response back to Transport

Retry deliveryGenerate an NDRReroute the message

Mail Delivery Flow

DAGMBX1

MBX Transport

Transport

DB2DB1

MBX2

MBX Transport

Transport

DB2DB1DB1 DB1

MAPI MAPI

SMTP

Summary Exchange Server 2013 uses Building Blocks to facilitate deployments at all scales – from self-hosted, small organizations to Office 365

Exchange Server 2013 provides you with an architecture that is Flexible, Scalable, and Simpler and helps you reduce costs

Related Content

EXL315 – Exchange Server 2013 High Availability and Site Resilience

Contact Me Later at scott.schnoll@microsoft.com

EXL332 – Exchange Server 2013 Deployment and Coexistence

EXL326 – The New Exchange – Data Loss Prevention

EXL333 – The New Exchange – Archiving and Compliance

Find Me Later at: 11:00am – 11:30am – Speaker Lounge on Wed | Thurs | Fri

Track Resources

Download Exchange Server 2013 Preview and Lync 2013 Preview

Try the new Exchange and Lync Online in the Office 365 Enterprise Preview

Test drive Exchange and Lync in our Customer Immersion Experience Centers

Get a Lync business value assessment or Exchange and Lync technical briefing

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.