Planning very large scale document repositories with high availability in share point 2013 v5

© 2010 WinWire TechnologiesWinWire Technologies, Inc. Confidential

Planning Very Large Scale Document Repositories with High Availability (HA) in

SharePoint Server 2013

@WinWire

Wired2Win Webinar Series

WinWire Technologies Inc.2350 Mission College Boulevard,Suite 480, Santa Clara, California 95054 Phone/Fax: +1 (888) 849 7339 Email : [email protected]

© 2010 WinWire TechnologiesWinWire Technologies, Inc. Confidential2


Who We Are

WinWire Technologies is a specialized IT solutions

company focused on helping business and technology

leaders achieve an “on-the-move” business environment

leveraging pre-built collaborative and analytic solution accelerators and

cross-platform mobile technologies.


This is the story of Bob.


Why is Bob sad?


Bob deployed SharePoint.


Things were pretty good.


So what’s the problem?


He needs to provide the following functionality to his organization.

Very Large Scale Document

Repositories

Highest possible availability

Predictable “Elastic”

architecture that can grow with the

deployment

Lowest possible maintenance and operations costs


He’s got many folks to please, including IT Support, legal and compliance.


Users are concerned.They don’t want it to be hard.


SharePoint seems to provide the right capabilities!


Or does it?


Bob isn’t sure how to proceed.


Let’s help Bob withPlanning Very Large Scale Document Repositories with High Availability (HA) in SharePoint


Agenda


Agenda


Enterprise Content Management

Create Control Protect

Create and organize content easily

Manage content policy, information architecture and taxonomy

Reduce risk and manage compliance with centralized tools


• Motivation• End-user experience is what really

matters.• Tail end of performance is sticky.

• Basic Idea• Categorize applications into SLA

buckets.

• Homogenize front-end, backend, and data layers.

• Prevent proliferation of app server deployments.

SharePoint 2013 Designing Your Farm


Designing for better SLA

Latency Throughput Resource Utilization

DesiredBehavior

Very low < 5 msec

Very High MediumConsistent Latency

Low <500 msec

Medium Low-MediumConsistent Latency

High> 1 min

HighHigh-VeryHigh

High Throughput

Very Low < 5 msec

Very High Low-MediumConsistent Latency

Front-End

Batch Processing Layer

Specialized Workload

Search

Database Layer

Request Management

Distributed Cache

Latency Throughput Resource Utilization

DesiredBehavior

Very low < 5 msec

Very High MediumConsistent Latency

Low <500 msec

Medium Low-MediumConsistent Latency

Low <500 msec

Medium Low – HighFairly Consistent Latency

High> 1 min

HighHigh-VeryHigh

High Throughput

Very Low < 5 msec

Very High Low-MediumConsistent Latency


< 10K Users

More Workloads

More Resources

< 10M Items

<100 Users

Evaluation

Very Light And Simple Workload

< 1,000 Users

Simple Workload

Small Content

Fault Tolerance

More & More Users

More Workloads

More Usage

Large Content

Multiple Index Partitions

Advanced Routing

Routing &

Caching

Front-End

Batch

Processing

Database

Crawl

Query

Search Specialized

Schematic Diagram…Machines could be virtual or physical.Topologies are rough estimates.

Evolution of topology


WORKLOAD

Facts: purpose, services, # of users, concurrency, rps, location

Tuning options: satellite farms, disable services, block clients …

Facts: #of site collections, DBs, data size

Tuning Options: split site collections, balance content…

Facts: # of servers, hw spec, roles, network

Tuning Options: add server, scale-up, tune settings …

What defines your SharePoint?


Microsoft’s 2013 Dogfood Farm

WORKLOAD

DATASET

HARDWARE

SLAs (During Peak Hours)1.3 TB total dataset size

1,001,141 documents

10 content DBs

Largest content DB - 290 GB

8,297 Site collections

Largest site collection - 275 GB

15,000 users

8,800 active users per week

1.7M requests per day

Collab, Social, Doc Mgmt, Project

204,106 profiles in UPA

1 Web App

Latency(99th ntile)

511 ms

RPS

50

Active Users (hr)

2,500

Availability

99.9%

Role HW Spec

Routing & Caching VM, 4 cores, 14GB RAM

Front End VM, 4 cores, 14GB RAM

Batch Processing VM, 4 cores, 14GB RAM

Database 8 cores, 64 GB RAM

Role Avg CPU/ Mem

Routing & Caching 12% / 8 GB

Front End 45% / 11GB

Batch Processing 80% / 12 GB

Database 11% / 46 GB


Capacity Planning Resources


Component CPU Network Disk Memory

Search administration 1

Crawler 2

Content processing (CPC)

3

Analytics processing (APC)

3

4

Index

Query processing (QPC) 5

1. Admin: Memory footprint increase with topology size

2. Crawler: Crawled documents are temporarily stored to

disk

3. CPC and APC are scheduled with “below normal”

priority

4. APC: Network bandwidth increase with scale out, but

only in-between APC nodes

5. QPC: Network IOPS increases linearly with number of

index partitions and QPS

Components – Scaling Cheat sheet


Sample Worksheet

Current Size Recommended storage

Notes

Configuration 2 GB The configuration database grows very slowly. Approximate growth is 40 MB per 50,000 Site Collections

Central Administration Content

1 GB Very slow growth. Not expected to grow beyond 1 GB.

Content (one or more) ????????????

• Functional Area (PROD) • Portal and Self Service• Legacy Data to be migrated. When migration of legacy data is performed, database sizing and

Site Collection design has to take place.• Cross Functional Area Collaboration

Usage and Health Data 10 GB Dependent on logging level selected

Subscription Settings Service 5 GB Dependent on number of alerts created by users

Search Administration 10 GB Search Administration is typically small

Crawl (one or more) 50 GB Crawl = 0.046 x (sum of content databases)

Property (one or more) 15 GB Property = 0.015 x (sum of content databases)

Profile 10 GB Approximately 1 MB per user profile

Synchronization 5 GB Approximately 650 kb per user profile

Social Tagging 2 GB Approximately 0.009 MB per tag

Staging 5 GB Web Analytics Service

Reporting 5 GB Web Analytics Service

Secure Store 1 GB

Word Automation Services 1 GB

Managed Metadata 10 GB The size of the database is affected by the number of content types and keywords.

Total ???


Agenda


Results from SharePoint 2013 Deployments

• 50% faster server response

• 4 X faster profile sync. • 10 X faster w/ Active Directory import

• 40% more efficient use of bandwidth

• 80% less file i/o related SQL IOPS

• More scalable SharePoint farms


SharePoint Farms Scale More than Ever

Product does have its limits

Limits are for each

Dimension

Each dimension needs to

stay within limits


SharePoint 2013 Performance Benchmarks

SharePoint 2010 benchmark on SharePoint 2013

SharePoint 2013

SharePoint 2010

4 front-end VMs w/ 4 CPUs, 12 GB RAM Green Zone: < 60% CPU on FEs, < 0.5 sec server response time

1 physical SQL Server w/ 8 CPUs, 32 GB RAM Max Zone: 90+% CPU on FEs, < 1 sec server response time


SharePoint 2013 Scale Divisional Benchmark

Collaboration + social + communities + meta-data navigation

Green Zone: < 60% CPU on FEs, < 0.5 sec server response time

Max Zone: 90+% CPU on FEs, < 1 sec server response time


Agenda


SharePoint Security Model: Conceptual

Token

TOKENUser

GroupsRoles

Claims

AuthenticationProvider

Identities

STSUserDS

AuthenticationMethod

Authorization

User

GroupRole

AssignmentPermission

Level

F D

RC

Security Scope

Site

Policies

ANONYMOUSACCESS & POLICY

USER POLICY

Web App

PermissionsGroupDS

Identity Authentication


End User Security Recommendations - Inheritance

34

• Roles assignments are inherited by child objects• Subsite, list, library, folder, item or document

• Best Practice• Use inheritance wherever possible

Role Assignment

Role Assignment

Site

W

Library


End User Security Recommendations - Inheritance

35

● Roles assignments are inherited by child objects– Subsite, list, library, folder, item or document

Role Assignment

Role Assignment

Site

W

Library

Scope 5

AD Group X (Reader)

User 3 (Contributor)

User 4 (Full Control)

Scope 3


Scope 4


Web Object

Document Library Object

Folder Object

Item 1 Object

Item 2 Object

Item 3 Object

Scope 2

User 5 (Reader)

+ User 1 (Limited Access)


1

1

1

1

1

1

SPGroup Object FullGP

+ User 3

+ User 4

SPGroup Object ContributeGP

+ User 1

+ User 2

SPGroup Object ReaderGP

+ User 5

+ AD Group X

Scope 1

+ FullControlGP (Full Control)

+ ContributorGP (Contributor)

+ ReaderGP (Reader)


You can break inheritance– Assign explicit permissions– Dynamic security changing code redesign– Recommendation is to modify scope architecture so

that scope membership does not cause ACL recalculation at the parent document library and Web. Only possible using custom code.

Role Assignment

Role Assignment

Site

W

Library

X

End User Security Recommendations - Fine-grained Permissions


End User Security Recommendations - Fine-grained Permissions

37

Role Assignment

Role Assignment

Site

W

Library

X

Scope 2

User 5 (Reader)

Scope 5

AD Group X (Reader)



Scope 3


Scope 4


Scope 1

User 2 (Reader)


User 6 (Contributor)Web Object

Document Library Object

Folder Object

Item 1 Object

Item 2 Object

Item 3 Object

+ AccessGP1 (Limited Access)

1

2

3

4

1

5

SPGroup Object AccessGP1

+ AD Group X

+ User 3

+ User 4

+ User 5

+ User 1

+ User 2




Agenda

38


Production Farm Disaster Recovery Architecture

39


Disaster Recovery Options

Standby Type

Definition Description Pros Cons

Hot Standby A second data center that can provide availability within seconds or minutes.

Multiple data centers

Content and services through only one data center

Relatively fast to recover Can be quite expensive to configure and maintain

Warm Standby

A second data center that can provide availability within minutes or hours

Virtual server images shipped to local and regional disaster recovery farms

Relatively inexpensive to recover, because a virtual server farm can require little configuration upon recovery

Can be very expensive and time consuming to maintain

Cold Standby

A second data center that can provide availability within hours or days

Backups are shipped to support bare metal recovery to local and regional offsite storage on a regular basis, and contracts are in place for emergency server rentals in another region

Often the cheapest option to maintain, operationally

Often an expensive option to recover, because it requires that virtual servers be configured correctly after a disaster has occurred

The slowest option to recover


Agenda


High availability (HA) with RBS

42

Size (GB)

ReductionWithout RBS

With RBS

Database size (100 GB) 217.2 7.0 96.8%

Database data file size

(100 GB) 106.9 3.2 97.0%

Database transaction log

file size (100 GB) 111.6 3.8 96.6%

Size of RBS externalized

data -- 96.2 --

Database size (1 TB) 2,292 26 98.9%

Database data file size (1

TB) 1,120 6.5 99.4%

Database transaction log

file size (1 TB) 1,173 20 98.3%

Size of RBS externalized

data -- 1,115 --

Effects of RBS on SQL Server Database Size

© 2010 WinWire TechnologiesWinWire Technologies, Inc. ConfidentialWinWire Technologies, Inc. Confidential

Topic: Line of Business (LOB) + Enterprise Search

When: December 05, 2013

Time: 8AM PST

Presented by: Bill Baer and Cem Aykan - Microsoft

Q & A

Next Webinar

© 2010 WinWire TechnologiesWinWire Technologies, Inc. ConfidentialWinWire Technologies, Inc. Confidential

Reach out to us at [email protected]

Visit us: www.winwire.com

http://www.winwire.com/winwire-blog/

www.twitter.com/winwire

Thank You

mailto:[email protected]

http://www.winwire.com/

http://www.winwire.com/winwire-blog/

http://www.twitter.com/winwire