Exchange Server 2013

ArchitectureRoss Smith IV

Principal Program Manager, Exchange Server

Disclaimer© 2012 Microsoft Corporation. All rights reserved.

Microsoft, Office 365, and other product and service

names are or may be registered trademarks and/or

trademarks in the U.S. and/or other countries.

The information herein is for informational purposes

only and represents the current view of Microsoft

Corporation as of the date of this

presentation. Because Microsoft must respond to

changing market conditions, it should not be

interpreted to be a commitment on the part of

Microsoft, and Microsoft cannot guarantee the

accuracy of any information provided after the date of

this presentation. MICROSOFT MAKES NO

WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS

TO THE INFORMATION IN THIS PRESENTATION.

Agenda

Discuss the new server

architecture in Exchange 2013

Agenda

Discuss the Client Access

server role

Discuss the Mailbox

server role

AD

5 major roles

Tightly

coupled

• Functionality

• Geo affinity

• Versioning

• User partitioning

Web

browser

Outlook

(remote user)

Mobile

phone

Line of

business

application

Client Access

Client connectivity

Web services

Outlook (local user)

Layer 7 LB

External

SMTP

servers

Hub Transport

Routing & policy

Forefront

Online

Protection for

Exchange

Mailbox

Storage of

mailbox items

Enterprise Network Phone system

(PBX or VOIP)

Unified Messaging

Voice mail and

voice access

Edge Transport

Routing and AV/AS

E2007/E2010 Server Role Architecture

What’s wrong with the existing model?

• Exchange deployments are overly complicated

• Doing Exchange load balancing “right” is hard and often requires

expensive solutions

– Requiring session affinity at the load balancer significantly impacts scalability

– Many hardware load balancing solutions are expensive and thus, are a luxury

many of our customers can’t afford or don’t have the expertise to deploy

• Customers deploy based on dedicated server roles

– This means that in many cases, hardware is deployed that is unutilized (e.g.,

DIMM slots and disk slots, etc.) or under-utilized

• Too many namespaces are required (especially in site resilient designs)

L7 LB

CAS HT

MBX MBX

LB

ExEx

Ex Ex

SAN

The EvolutionExchange:

Separate roles for ease of deployment and mgmt. segmentation

Support cheaper storage

2007

Simplify for scale, balanced utilization, isolation

Integrate HA for all roles

Simplify network architecture

2013

Separate HA solutions for each role

Introduced the DAG

Rich management experience using RBAC

Leaves resources on the ground in each role

2010

Role differentiation through manual configuration

Hardware solutions for “reliability” ($$$$)

ExEx

Ex Ex

SAN

2000/2003

L7 LBCAS HT

MBX MBX

LB

Exchange: The Evolution

The new server role

architecture

Exchange 2013 Architecture Theme

Use Building Blocks to facilitate deployments at all scales – from self-hosted small organizations to Office 365

– Server role evolution

– Network layer improvements

– Versioning and inter-op principles

Benefits– Hardware efficiency

– Deployment simplicity

– Low friction cross-version inter-op

– Failure isolation

AD

Web

browser

Outlook

(remote user)

Mobile

phone

Line of

business

applicationOutlook (local user)

External

SMTP

servers

Forefront

Online

Protection for

Exchange

Enterprise Network

Phone system

(PBX or VOIP)

Edge Transport

Routing and AV/AS

2 Building BlocksClient Access Array

• Evolution of E2010 CAS Array

• SMTP Front-End

Database Availability Group

• Evolution of E2010 DAG

• Includes core server protocols

Loosely coupled• Functionality

• Versioning

• User partitioning

• Geo affinity

Layer

4LB

CAS

CAS

CAS

CAS

CAS

CAS Array

MBX

MBX

MBX

MBX

MBX

DAG

Exchange 2013 Server Role Architecture

E2010Banned

Every Server is an Island

Server1 (Vn) Server2 (Vn+1)

Protocols,

Server Agents

EWS

RPC CA

Transport

Assistants

MRS MRSProxy Transport

Assistants

EWS

RPC CA

MRS MRSProxy

Business Logic

XSO Mail Item

Other APICTS

XSO Mail Item

Other APICTS

Storage

Store Content index

File systemESE

Store Content index

File systemESE

SMTP

MRS proxy protocol

EWS protocol

Custom WS

E2013 Architecture

Functional Layering

Hardware LB

CAS, HT, UM

MBX

L7LB

E2010 Architecture

AuthN, Proxy, Re-direct

Protocols, API, Biz-logic

Assistants, Store, CI

Load Balancer

AuthN, Proxy, Re-direct

CA

S2

01

3

Store, CI

Protocols, API, Biz-logic

MB

X2

01

3

CAS

The key to enlightenment…

For a given mailbox’s connectivity, the

protocol being used is always served by

the protocol instance that is local to the

active database copy

This means that the rendering for clients like

OWA occurs on the Mailbox server

This means that Transport transcoding is

occurring on the Mailbox server etc…

User

DAG1

MBX-A MBX-BMBX-BMBX-A

Client Access Protocol

Proxying

What is the CAS2013 role?• CAS2013 is comprised of three components:

– Client protocols (HTTP, IMAP, POP)

– SMTP

– UM Call Router

• Thin, stateless (protocol session) servers organized in a load balanced configuration– Session affinity NOT required at the load balancer

• Provides a unified namespace and authentication for clients

• Where the logic “lives” to route a specific protocol request to the “correct” destination end point– Capable of supporting legacy servers with redirect or proxy logic

• Is a domain-joined machine in the corporate forest

CAS2013 Client Protocol Architecture

CAS2013

MBX2013

RPC CA

IIS

RPS OWA, EAS, EWS, ECP, OAB

POP

IMAPTransport UM

RpcProxy

MDB MailQ

HTTP Proxy

IISPOP

IMAPSMTP UM

TelephonyIMAP SMTP OWA EAS EACOutlook PowerShell

Load

Balancer

HTTP POP

IMAPSMTP

Redirect

SIP

+

RTP

Outlook ConnectivityRPC/HTTP and the death of RPC/TCP

• What are the benefits?

– Does not require a “RPC CAS array namespace” for the DAG

– No longer have to worry about “The Exchange administrator has made a

change that requires you to quit and restart Outlook” during mailbox

moves or *over events

– Extremely reliable and stable connectivity model – the RPC session is

always on the MBX2013 server hosting the active database copy

• What changes?

– RPC end point for Outlook client is now a GUID (and SMTP suffix)

– Support for internal and external Outlook Anywhere namespaces

RPCProxy.dll

Third-Party MAPI Products

• Third-party MAPI products will need to use RPC/HTTP to connect to

CAS2013

• Exchange 2013 will be the last release that supports a MAPI/CDO

download

– Third-parties must move to EWS in the future

• The MAPI/CDO download will be updated to include support for

RPC/HTTP connectivity

– Will require third-party application configuration; either by

programmatically editing a dynamic MAPI profile or setting registry keys

– Legacy environments can continue to use RPC/TCP

CAS2013 Client Protocol Connectivity Flow

MBX

CAS

Load Balancer

HTTP Proxy

IIS

DB

Protocol Head

Local Proxy Request

HTTP

HTTP

Site

Bo

un

dary

MBX

CAS

Load Balancer

HTTP Proxy

IIS

DB

Protocol Head

HTTP

OWA Cross-Site Redirect Request

HTTP

MBX

DB

Protocol Head

HTTP

Cross-Site Proxy Request

HTTP

Site

Bo

un

dary

Generalist IT admin Those with increased

network flexibility

Those who want to

maximize server

availability

+ Simple, fast, no affinity LB

+ Single, unified namespace

+ Minimal networking skillset

- Per Server Availability

+ Per protocol availability

+ Single, unified namespace

- SSL termination @ LB

- Requires increase networking

skillset

+ Simple, fast, no affinity LB

+ Per protocol availability

- One namespace per protocol

Simplicity

FunctionalityWho’

s it fo

r?Tr

ad

e-O

ffs

Exchange Load Balancing Options

A Single Common Namespace ExampleGeographical DNS Solution

Sue (somewhere in NA)

DNS Resolution

DAG

VIP #1 VIP #2

Sue (traveling

in APAC)DNS Resolution via Geo-DNS

Round-Robin between # of VIPs

DAG

VIP #3 VIP #4

mail.contoso.com

Round-Robin between # of VIPs

CAS2013 Client Protocol Benefits• Simplifies the network layer

– No longer requires session affinity at the load balancer

– Just get the traffic to CAS2013 and let it handle the affinity

– CAS2013 can be “farther away” from MBX2013 and still offer good client performance (because it is a 1:1 proxy)

– Removes the need for RPC Client Access arrays

• Deployment flexibility– CAS2013 provides more deployment flexibility; for example, consolidate to

fewer sites

– Can deploy a single world-wide namespace

• Simplifies upgrade and inter-op– Designed to proxy to multiple Mailbox server versions, up and down level

– DAGs can be replaced with E2013 at any desired pace

The Mailbox Server Role

The Mailbox Server Role

• A server that hosts all the components that process, render and store the data

• Clients do not connect directly to MBX2013 servers; connectivity is through CAS2013

• Evolution of E2010 DAG

– Collection of servers that form a HA unit

– Databases are replicated between servers in a given DAG

– Servers can be in different locations, for site resiliency

– Maximum of 16 Mailbox servers

– 50 database copies / server

MBX1

MBX2

MBX16

The New Store Process

• Store is effectively made up of three processes

– Replication service

– Store service process/controller

– Store worker process

• Replication service initiates failovers and is responsible for issuing

mount/dismount operations

• Store service process/controller manages the store worker processes

• Each database has its own Store worker process

Exchange IOPS Trend

DB IOPS/Mailbox

Exchange 2003 Exchange 2007 Exchange 2010 Exchange 2013

1

0.8

0.6

0.4

0.2

0

+99% reduction!

Large Mailboxes for the win! Large Mailbox Size 100GB+

– Aggregate Mailbox = Primary Mailbox +

Archive Mailbox + Recoverable Items

– 1-2 years of mail (minimum)

Increased knowledge worker

productivity

Eliminate or reduce PST reliance

Eliminate or reduce third-party

archive solutions

Outlook 2013 allows you to control

OST size!

– Gives more options around mailbox

deployments

1 Day 150 11 MB

1 Month 3300 242 MB

1 Year 39000 2.8 GB

2 Years 78000 5.6 GB

4 Years 156000 11.2 GB

New Exchange Search Infrastructure

• Leverages Search Foundation

– Common, actively developed search platform used across Office server

products

– Does consume more memory (1/6 available memory) to improve query

performance

• Provides

– Significantly improved query performance compared to E2010

– Significantly improved indexing performance compared to E2010

• Feature parity with E2010 search

• Leverages the same cmdlets like Get-MailboxDatabaseCopyStatus

Exchange IndexingReduced Processing of Body and Attachments

MBX2013

Transport

Mailbox

DB Idx

ExSearch CTSStore Index Node

Transport Content Transformation

Service

Local Delivery

Log

Reliable

EventRead

Content

MBX2013

Mailbox

DB Idx

Passive

Log

Public FoldersDawn of a New Age

Architectural betPublic folders are based on the mailbox architecture

Details• Hierarchy is stored in PF mailboxes (one writeable)

• Content can be broken up and placed in multiple mailboxes

• The hierarchy folder points to the target content mailbox

• Uses same HA mechanism as mailboxes

• No separate replication mechanism

• Single-master model

• Similar administrative features to current PFs (setting quota, expiry, etc.)

• No end-user changes (looks just like today’s PFs)

Not all public folder usage scenarios are best served by public folders

MBX2013

CAS 2013

MBX2013 MBX2013

Private

logonPublic

logon

Content MailboxHierarchy Mailbox

Public Logon

Transport Architecture

Transport Components on Client AccessFront-End Transport Service

• Handles all inbound and outbound external SMTP traffic for the

organization, as well as client endpoint for SMTP traffic

– Does not replace the Edge Transport Server Role

• Functions as a layer 7 proxy and has full access to protocol

conversation

• Will not queue mail locally and will be completely stateless

• All outbound traffic appears to come from the CAS2013

• Listens on TCP25 and TCP587 (two receive connectors)

Processing Inbound MessagesExterna

l Server

CAS MBX 1. New SMTP Connection

2. CAS performs envelope filtering

3. CAS determines route to best MBX

server

4. Message delivery begins

1. If successful, CAS returns 250 OK

acknowledgement to external

server

2. If unsuccessful, CAS returns 421

response

Benefits of SMTP Front-End Service

• The SMTP Front-End Service provides:

– Protocol level filtering – performs connection, recipient, sender and

protocol filtering

– Network protection – centralized, load balanced egress/ingress point for

the organization

– Mailbox locator – avoids unnecessary hops by determining the best

MBX2013 to deliver the message

– Load balanced solution for client/application SMTP submissions

• Scales based on number of connections – just add more servers

Transport Components on Mailbox

• Transport in MBX2013 has been broken down into three components

– Transport Service - Stateful and handles SMTP mail flow for the organization

and performs content inspection (Was previously referred to as “Hub

Transport”)

– Mailbox Transport Delivery Service - Receives mail from the Transport service

and deliveries to the Mailbox Database

– Mailbox Transport Submission Service - Takes mail from the Mailbox Databases

and submits to the Transport service

• Mailbox Transport is stateless and does not have a persistent storage

mechanism

• Mailbox Transport performs content conversion

Transport Components on MailboxResponsibilities

• Receives all inbound mail to the organization (Proxied through CAS

or direct)

• Submits all outbound mail from the organization (Proxied through

CAS or direct)

• Handles all internal message processing such as Transport Rules,

Content Filtering, and Anti-Virus

• Performs mail flow routing

• Queue messages

• Supports SMTP extensibility

Routing Optimizations• Next hop selection is broken down into distinct delivery groups:

– Routable DAG

– Mailbox Delivery Group

– Connector Source Servers

– AD Site (Hub Sites; Edge Subscriptions)

– Server list (DG expansion servers)

• Queuing is per delivery group, connector, or mailbox

• Once message is received at final destination, Transport will deliver the message via SMTP to Mailbox Transport on the server hosting the active database copy

• Send/Delivery-Agent Connectors can have source servers from multiple DAGs or AD Sites, and can be proxied through CAS

Mail Delivery

CAS /

MBX

MBX-1

DB2DB1

Transport

Mailbox

Transport

MBX-2

DB2DB1

Transport

Mailbox

Transport

DAG

SMTP

SMTP

MAPI

SMTP

Service Availability

Improvements

Service Availability Improvements

• Best copy selection now includes health of services when

selecting best copy

• Failover time reductions

• Lagged copy enhancements

• Simpler site resilience strategy

• Managed Availability

Managed Availability• Monitoring and recovery

infrastructure is integrated with

Exchange’s high availability

solution

• Detects and recovers from

problems as they occur and are

discovered

• Is user focused – if you can’t

measure it, you cannot monitor it

Managed Availability

—OWA send

—OWA failure

—OWA failure detected

—OWA restart service

—OWA restart complete

—OWA verified as healthy

—OWA send

—OWA failure

—OWA failure detected

—OWA restart service

—OWA restart service failed

—Failover server’s databases

—OWA service restarts

—OWA verified as healthy

—Server becomes “good” failover target (again)

LB CAS-1

CAS-2

DAG

MBX-1

DB1 DB2

MBX-2

OWA DB1 DB2

MBX-3

OWA DB1 DB2

OWAOWAOWAOWA DB1

DB1

Managed Availability + Retries…“stuff breaks and the Experience does not”

Transport High Availability Improvements

• Every message is redundantly persisted before its receipt is

acknowledged to the sender

• Delivered messages are kept redundant in transport similar to

active messages

• Every DAG represents a transport HA boundary and owns its

HA implementation

– If you have a stretched DAG, you also have transport site resilience

• Resubmits due to transport DB loss or MDB *over are fully

automatic and do not require any manual involvement

Transport HA

Site

Bo

und

ary

MBX1

Transport

MBX Transport

Store

DB

1

DB

2

Mail.que

MBX2

Transport

MBX Transport

Store

DB

1

DB

2

Mail.que

MBX3

Transport

MBX Transport

Store

DB

1

DB

2

Mail.que

MBX4

Transport

MBX Transport

Store

DB

1

DB

2

Mail.que

MBX5

Transport

MBX Transport

Store

DB

3

DB

4

Mail.que

MBX6

Transport

MBX Transport

Store

DB

3

DB

4

Mail.que

MBX7

Transport

MBX Transport

Store

DB

3

DB

4

Mail.que

MBX8

Transport

MBX Transport

Store

DB

3

DB

4

Mail.que

CAS2013 or

MBX2013SMTP250 OK

250 OK

250 OK

250 OK

R1, R2, R3

R3R3

R3R2R1

R1, R2, R3

Log Log LogLog Log Log

Log Log Log

1. Maintain a copy of the message in the queue database but don’t acknowledge the DATA verb

2. Generate a shadow copy on another MBX2013 server in the DAG (remote site preferred)

3. Wait for acknowledgement from the shadow server4. Send acknowledgement to SMTP client5. Delete message from queue after SafetyNet

threshold has expired

– Facilitates deployments at all scales – from self-hosted small organizations to Office 365

– Provides more flexibility in namespace management

– All core Exchange functionality for a given mailbox is served by the MBX2013 server where that mailbox’s database is currently activated

– Simplifies the network layer

– Transport protection is built-in

– All components in a given server upgraded together

– No need to juggle with CAS <-> MBX versions separately

– Utilize CPU core increase, cheaper RAM

– Utilize capacity effectively

– Fewer disks/server => simpler server SKUs

Thank you!