Design, performance and capacity - Factors of successful intranet

and internet SharePoint sites

Sanjay Narang,Anandan Kumaran,

Singaram Murigappan

Agenda

• Real Life Implementation – www.cognizant.com

• Real Life Case Study – Capacity Planning

• Aspects Of Performance• Capacity Management Model–Model– Design

www.cognizant.com

SharePoint 2010 Implementation

Publish Content

• Marketing team needs a robust mechanism to publish content effectively without needing specialized technical skill

Content Deployment

Content authored is saved in the authoring server database

Content push by marketing admin

Content authored is moved to the preproduction environment. It allows the user to verify the content before pushing live

Content push by marketing admin

Content is moved from preproduction environment to live environment after verification

Authoring Pre-Production www.cognizant.com

Intuitive User Interface

Intuitive User Interface

Content Authoring

• nt

Content Author needs a mechanism to add/edit content, upload documents without worrying about the layout or having pre-requisite design skills

Author Content

Edit Page

Edit Page Contd.,

Edit Page Contd.,

Our Approach – Edit View

Public View

Friendly URLs

Need for friendly URLs that are easier for users to remember and for search engines to find

http://www.cognizant.com/banking-financial-services/Pages/landingPage.aspx

URL Rewrite

• Powerful rules to transform complex URLs into simple and consistent web addresses

• Rule-based URL rewriting engine

• Regular expression pattern matching

• Downloadable

• Download:http://go.microsoft.com/?linkid=9722532

Configuration - Inbound

• Inbound rules convert the user friendly URLs provided by the users to actual URLs

Configuration - Outbound

• Outbound rules convert the actual URLs to the user friendly URLs

Friendly URL - Output

http://www.cognizant.com/banking-financial-services

URL Rewrite Support

Not officially supported by the Microsoft Product Team

Faster Site

Need for the site to equally responsive and quick to all users across the globe. The load of users on the application will vary especially during financial results but I need a cost effective solution

SharePoint Out-Of-Box

• Output Caching• Object Cache• This can load but not latency

Content Delivery Network

• Deliver content faster• Scalability• Across the globe users has similar

experience on accessing the site

Capacity Planning and Management

Case Study – Contoso Intranet Portal

• 150,000 users spread across globe• Existing site – The gateway to all other applications– primarily HTML/ASP/JavaScript

• New site– Intranet Publishing on SharePoint Server

2010– Widget based homepage, – Much more content on home page including

Flash videos

Capacity Management

• Capacity Planning– One time sizing– Selection &

Configuration• Data architecture• Topologies (physical

and logical)• Hardware

• Capacity Management– Extends Planning– Cyclic Process– Monitored &

Optimized– Changing

Conditions and Requirements

Analogy

Analogy

Capacity Management Model

Aspects of Performance

• Latency• Throughput • Data Scale• Reliability

capacity management

Model

Model

Design

Pilot, Test,

OptimizeDeploy

Monitor &

Maintain

Definitions and Conventions

• Users– Concurrent Users– Active Users

• RPS

Definitions and Conventions

• User Types

User load Request rate per Hour Seconds for a request per user

Light 20 180

Typical 36 100

Heavy 60 60

Extreme 120 30

Model

• Determine Workload• Determine DataSet • Set Performance and Reliability

Targets

Model

RPS Estimation (New Env)

Number RPH/User RPHTotal Users 150,000Active Users 50,000

Light (10%) 5,000 20 100,000 Typical (70%) 35,000 36 1260,000 Heavy (15%) 7,500 60 450,000 Extreme (5%) 2,500 120 300,000

Total RPH 2,111,000RPS (RPH/3600) 586Concurrency 50%

Effective RPS 293

Model

• IIS Logs– Take from all servers into a central

location– Take it for one of the peak day

• User Tools – LogParser 2.2

RPS Calculations (Existing Env)Model

Get RPS per second

logparser -i:IISW3C -o:CSV "SELECT QUANTIZE(TO_TIMESTAMP(date, time), 3600) AS Hour, QUANTIZE(TO_TIMESTAMP(date, time), 1) AS Second, COUNT(*) AS RPS

INTO RPS_No401s.csv FROM *.log WHERE sc-status <> 401 GROUP BY Hour, Second"

Model

Hour Second RPS

6/29/2010 18:00 6/29/2010 18:30 10

6/29/2010 18:00 6/29/2010 18:30 5

6/29/2010 18:00 6/29/2010 18:30 11

6/29/2010 18:00 6/29/2010 18:30 55

--- --- ---

6/30/2010 4:00 6/30/2010 4:32 542

6/30/2010 4:00 6/30/2010 4:32 411

6/30/2010 4:00 6/30/2010 4:32 648

6/30/2010 4:00 6/30/2010 4:32 1450

6/30/2010 4:00 6/30/2010 4:32 1136

6/30/2010 4:00 6/30/2010 4:32 1274

6/30/2010 4:00 6/30/2010 4:32 1150

6/30/2010 4:00 6/30/2010 4:32 1178

6/30/2010 4:00 6/30/2010 4:32 890

6/30/2010 4:00 6/30/2010 4:32 556

6/30/2010 4:00 6/30/2010 4:32 710

--- --- ---

6/30/2010 18:00 6/30/2010 18:29 7

6/30/2010 18:00 6/30/2010 18:29 5

RPS Every Second for 24 Hours

Model

Hour AvgRPS MaxRPS

6/29/2010 18:00 27 229

6/29/2010 19:00 21 188

6/29/2010 20:00 20 178

6/29/2010 21:00 20 202

6/29/2010 22:00 16 153

6/29/2010 23:00 16 184

6/30/2010 0:00 22 168

6/30/2010 1:00 37 194

6/30/2010 2:00 141 646

6/30/2010 3:00 526 1361

6/30/2010 4:00 630 1450

6/30/2010 5:00 508 1261

6/30/2010 6:00 407 981

6/30/2010 7:00 332 988

6/30/2010 8:00 351 1016

6/30/2010 9:00 371 1163

6/30/2010 10:00 385 997

6/30/2010 11:00 475 1320

6/30/2010 12:00 350 977

6/30/2010 13:00 213 641

6/30/2010 14:00 122 439

6/30/2010 15:00 86 436

6/30/2010 16:00 58 345

6/30/2010 17:00 43 338

6/30/2010 18:00 30 213

Avg and MaxRPS Every Hour for 24 Hours

Model

Avoid Image, CSS, JS

• Get File Extensionto_lowercaseEXTRACT_EXTENSION(cs-uri-stem)) • exclude Where clausefext <> ‘jpg’ AND fext <> ‘jpeg’ ANDfext <> ‘gif’ AND fext <> ‘png’ ANDfext <> ‘js’ ANDfext <> ‘css’

Model

ModelDecision Point – Peak or

Average

• Average RPS on a day• Average RPS in a peak hour• Maximum RPS in a peak hour

220

700

1500

Dataset - Content DBS.No Parameter Value Units Source of Value

AExisting size of all documents (pages, images etc) without versions 40GB Actual size from web servers

BExisting size of all documents (pages, images etc) with all versions 160GB Calculated as in Table 1

CTotal Number of documents with versions 1,600,000 Calculated as in Table 1

DEstimated number of list items per document 2

Approximate assumption based on similar projects

E Total number of list items 3,200,000 C x D

FAvg size of a list item or metadata for a document 10KB

Approximate assumption based on similar projects

G Total size of list items 31GB (E x F)/(1024 x 1024)H Total size of document metadata 15GB (C x F)/(1024 x 1024)I Content DB size for existing content 206GB B+G+H

Model

Content DB Calculations

S.No Parameter Value Units Source of Value

I Content DB size for existing content 206GB B+G+H

JGrowth estimated over next 2-3 years 100% Approximate assumption

K Total size with growth 412GB I + (I x J)/100L Recycle Bin 30%M Total size with Recycle bin 535GB K + (L x K)/100

Model

All Databases Calculations

Parameter Value (in GBs)Content databases for Main Portal site collections 535Custom databases 10Configuration and CA databases 5

Database for Usage and Health data collection SA 200Web Analytics Databases 200Managed Metadata SA 10User Profile SA 300TOTAL 1,260

Model

Overall SQL Server Storage

Storage For Value (GB) %age of DBDatabases 1,260Backup 1,890 150%System 10Transaction Log 252 20%temp db 252 20%Total for Databases 3,664Empty Space (25% of total) 916TOTAL for SQL Server 4,580

Model

Performance and Reliability Targets

• Availability– Numbers of Nines (99%, 99.9%, 99.99%)– Consider• Patch Management• Code Releases

• Server Responsiveness– E.g. < 1 sec

• System Resource Utilization– E.g. Avg CPU < 50% , Available Memory

Model

Model - Summary

• Total Number of Users – 150,000• Total Request in a day – 18,000,000• Throughput (RPS) – 1500, 700, 220• Storage - 5 TB• Server Response Time < 1 sec

Model

capacity management

Design

Model

Design

Pilot, Test,

OptimizeDeploy

Monitor &

Maintain

Design

• Identify case study• Choose an Architecture• Determine Hardware

Design

Know SharePoint 2010

• Vs SharePoint 2007– Service Applications & Cross Farm

Publishing

• Oversizing vs. Undersizing• Reference Architectures– Single server, small, medium and large

farms

Design

Constraints and Boundaries

• Site Collection < 100 GB• Content DB < 200 GB• 1 SQL Instance < 5 TB• Content DBs per Web App < 300

Design

Reference ArchitecturesDesign

Reference Architectures

Identify Case StudyDesign

Case Study - Published Intranet

Workload Characteristics Value

Average Requests per Second (RPS) 100

Average RPS at peak time 226

Total number of unique users per day 33,580

Average concurrent users 172

Maximum concurrent users 376

Total # of requests per day 3,800,000

Design

DesignC

ase S

tud

y -

Pu

blish

ed

In

tran

et

Lab - RPS with Authentication Design

Be Aware of Thumb Rules

• Number of Servers – 1 WFE per 10 K users– 1 SQL server instance per 4 WFEs• 3-5 WFE cores per SQL core

– 1 DC per 3 WFEs

• RAM– 2-4 GB memory CPU core

Design

Choosing an Architecture

• High Availability– Hardware vs Software Load Balancer for

web servers– Clustering or Mirroring for SQL Server

• Service Applications – Separate farm or same farm

• Search Target– Dedicated or Distributed

Design

Determining Hardware Server Type Minimum (as per

TechNet)My Recommendation

CPU Web / App 64-bit, 4 cores 64 bit, 8 cores

DB 64-bit, 4 – 8 cores 64-bit, 16 cores

Memory Web/App 8 GB 16 GB

DB 8-32 GB 16-32 GB

Disk Web/App 80 GB 200 + 150 GB

DB As per requirement

IOPS Web/App Minimal

DB As per requirement

Design

SQL Server RAM Recommendations

Design

Combined size of content databases Recommended RAM Minimum for small production deployments 8 GB

Minimum for medium production deployments 16 GB

Recommendation for up to 2 terabytes 32 GBRecommendation for the range of 2 terabytes to a maximum of 5 terabytes 64 GB

RAID Levels & IOPS

• RAID 10

Design

Type RAID level Spindles Optimization TempDB [RAID 1+0] 2 IOPS/ GB Write optimized

Transaction Logs [RAID 1+0] 2 IOPS/GB Write optimizedContent databases

[RAID 1+0] (Can have RAID 5, if storage is a constraint)

0.75 IOPS / GB Read optimized

Design - Summary

• Workload – Published Intranet• Case Study – WCM Lab Results• Architecture– Availability• Hardware Load Balancer• Clustering for SQL Server

– Service Applications in separate farm

• Number of Servers• Hardware Configurations

Design

Design - Summary

• Number of Servers– Lab results: 4 servers = 3200 RPS, 2

servers = 1700 RPS– Scaled UP SQL server for 5 servers

• Hardware Configurations– RAM oversized because of low cost– Used thumb rule for SQL Memory

Design

Possible Variations

• Internet Website (anonymous)• Requirement for local services: Office

Web Apps, Excel Services etc.• Data Scale more than 5 TB

© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and

Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.