B365 saturday practical guide to building a scalable search architecture in sharepoint 2013

http://business365.azurewebsites.net

Practical Guide to building a scalable Search Architecture in SharePoint 2013 Thuan Nguyen, SharePoint MVPSolution ArchitectFPT Asia Pacific Pte. Ltd@nnthuan

twitter: #business365sg


About Me Microsoft SharePoint MVP Solution Architect at FPT

Asia Pacific with 6 years of experience with SharePoint platform.

Focus on building a comprehensive SharePoint enterprise-class solution

Email: [email protected] Blog: http://thuansoldier.net



What the talk is aboutCreating a search service application, adding content sources and performing full crawl are what you have ever heard about making search work for your SharePoint 2013. Imagine one day your boss comes to you asking about the need of a search for millions of documents. Adding more hardware resources would be flashed in your mind.

What is next? twitter: #business365sg


Agenda Common Misunderstandings Architecture & Topology Practical Guide Question & Answer



Common Misunderstandings For High Availability, create two Search Service Applications

There is only one machine playing Search role in your farm Scale out Search architecture by adding more servers. Start Search service is to make search functionality work.



Architecture & TopologyLogical Architecture

CrawlContent ProcessingAnalytics Processing

IndexAdministrationQuery Processing


Understand each component will help better design a scalable & maintainable Search for your organization.


Crawl Component


Responsible for crawling content from difference sources (by default)

SharePoint sites Exchange File Shares Line of Business Data HTTP Website Custom Repository

Deliver crawled items to content processing component.

Crawl database stores information about crawl items and crawl history


Content Processing Processes crawled items and

passes these items to the index component

Performs linguistic processing at index time (e.g. language detection and entity extraction)

Writes information about links and URLs to the Link database



Analytics Processing


Analyzes crawled items and how users interact with search results.

When an user does an action (e.g. view a page) the event is collected in usage files on the WFE’s and regularly pushed to event store where they are stored until processed

Results are then returned to the Content Processing Component to be included in the search index


Index Component Receives the processed items from the content processing

component and writes them to the search index. Handles incoming queries, retrieves information from the

search index, and sends back the result set to the query processing component.


An index partition is a logical portion of the entire search index.

Each partition is served by one or more index components (or “replicas”)


Query Processing


Analyses and processes search queries and results. The processed query is then submitted to the index

component, which returns a set of search results for the query.


Search Administration


Search Admin Component Is responsible for search provisioning and

topology changes Coordinates search components – Content

Processing, Query Processing, Analytics, and Indexing.

Search Admin DB Stores search configuration data:

Topology Crawl rules Query rules Managed property mappings Content sources Crawl schedules

Stores Analytics settings

Practical Guide

twitter: #business365sg http://business365.azurewebsites.net

Assessment Design Implementation Verification


Assessment


Don’t hastily touch your SharePoint. Leave it alone! Think about your content

What are your content sources (SharePoint document library, Exchange, File Server..)?

How much of content you want to search? (e.g. 100,000 documents)

Assess the number of concurrent users. Search database sizing High Availability requirement


Assessment


Sizing factor: Total Content Database Size Total Index Size (for index replica) Query Component Index Size Disk Storage Link Database Search Admin Database Total Crawl Database Size Total Crawl Database Log Size Analytics Database Size

ÞTotal database size for Search (except index replica)Contact me at [email protected] for search sizing calculator


Assessment


What is exactly High Availability for Search? Business language: Search doesn’t stop end users

searching something Technical language: All search logical components

and Search databases must be functional as always. Two or more Search service applications Add one more server and start search services on

two servers


Design


Don’t hastily touch your SharePoint. Leave it alone!

Start with one machine hosting all components


Design Don’t hastily touch your

SharePoint. Leave it alone!

Think about two machines for Search but different set of components


If one goes down, Query component in another machine still keeps functioning.


Design Don’t hastily touch your

SharePoint. Leave it alone!

Think about two machines for Search with the same set of components



Design


Don’t hastily touch your SharePoint. Leave it alone!

Do you need three machines for Search? Speed up Query

component? Reduce crawling time? Balance CPU utilization in

machine?


Design


Component CPU Network Disk RAM

Crawl Component MEDIUM HIGH MEDIUM MEDIUM

Content processing (CPC) HIGH MEDIUM HIGHAnalytics processing (APC) MEDIUM HIGH MEDIUM MEDIUM

Index Component HIGH MEDIUM HIGH HIGH

Query processing (QPC) MEDIUM MEDIUM MEDIUMSearch Admin Component LOW LOW LOW


Design


Volume of content Sample Search Architecture< 1 mil items Single-server Search farm 1 mil – 5 mil Two-server Search farm5 mil – 10 mil Small Search farm (3-4 servers)10 mil – 40 mil Medium Search farm (5-6 servers)> 40 mil Large Search farm


Sample Search Architecture


Handle number of different content sources (with 20 custom applications)

Nearly 1 million items Full crawl takes 2 hours Serving for nearly 20,000 users with

500 concurrent users.


Sample Search Architecture


Optimize search query to serve hundreds of concurrent users.

Handle 2-3 million of documents

No HA for Analytics and Admin


ImplementationCentral Administration doesn’t help much.PowerShell is your friend

1. Create Search Service Application2. Clone existing topology3. Modify Search component based on your designated

architecture4. Assign Index component and location5. Activate the new Search topology



PowerShell Scripts http://bitly.com/search_multi_server_PS http://bit.ly/search_2013_gui



Verification Central Administration can help


PowerShell Get-SPEnterpriseSearchStatus Get-SPEnterpriseSearchTopology


Helpful References SharePoint 2013: SharePoint and Enterprise Search

Survival Guide - http://bit.ly/search_survival_guide Plan enterprise search architecture in SharePoint Server

2013 - http://bit.ly/plan_for_ent_search Search Architecture for SharePoint 2013 -

http://zoom.it/Tsuy



Q & Atwitter: #business365sg


Please complete the evaluation

http://j.mp/b365-sg



Thank you to our sponsors!


Software

B365 saturday practical guide to building a scalable search architecture in sharepoint 2013