27
WHITE PAPER metalogix.com INTRODUCTION............................................................................................. 2 REPLICATOR PERFORMANCE AND SCALABILITY FEATURES .................. 4 Replicator Multi-Engine Deployment ................................................. 7 Multi-Threaded Replication Queue Architecture ............................. 7 Customized Replication Interval....................................................... 8 Configurable Package Event Count ................................................ 8 Configurable Package Event Processing Duration ..........................9 Replication Package Compression.................................................. 9 Hardware Network Compression Support ..................................... 9 Software Package Compression ................................................... 10 Remote Differential Compression (Rdc) ........................................ 10 Independent Outbound Event And Inbound Event Processing Controls....................................................................... 11 Selective Structure Replication ..................................................... 11 Rule-Based Item-Level Content Replication ................................. 11 Selectable Replication Events....................................................... 12 Shared Replication Packages ....................................................... 12 Package Database Caching.......................................................... 13 Configurable Replication Monitor Update Level ........................... 13 Improved Initial Replication Support ............................................. 14 MICROSOFT TECHNOLOGY CENTER PERFORMANCE AND SCALABILITY BENCHMARK............................................................... 14 METALOGIX NETWORK TEST LAB PERFORMANCE BENCHMARKING... 16 UNDERSTANDING REPLICATOR PERFORMANCE AND SCALABILITY ....................................................................... 17 Metalogix Replicator Pipeline ....................................... 17 Replicator Performance ...........................................18 Outbound Event Processing .................................. 19 Package Transfer ......................................................20 Inbound Event Processing ......................................21 Performance Monitoring .........................................22 Replicator Pipeline Performance Features .............24 Replicator Scalability ..............................................25 APPENDIX A – ABOUT WINDOWS REMOTE DIFFERENTIAL COMPRESSION ................................................................ 25 Copyright © 2012 Metalogix International GmbH. All rights reserved. Metalogix is a trademark of Metalogix International GmbH. StoragePoint is a registered trademark of BlueThread Technologies, Inc. Microsoft, Exchange Server, Microsoft Office, SharePoint, and SQL Server are registered trademarks of Microsoft Corporation. Metalogix Replicator for SharePoint ENTERPRISE PERFORMANCE AND SCALABILITY Version 4.1 June 2012 CONTENTS

Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

  • Upload
    others

  • View
    23

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

W H I T E PA P E R

m e t a l o g i x . c o m

INTRODUCTION............................................................................................. 2

REPLICATOR PERFORMANCE AND SCALABILITY FEATURES .................. 4

Replicator Multi-Engine Deployment ................................................. 7

Multi-Threaded Replication Queue Architecture ............................. 7

Customized Replication Interval....................................................... 8

ConfigurablePackageEventCount................................................8

ConfigurablePackageEventProcessingDuration..........................9

ReplicationPackageCompression..................................................9

HardwareNetworkCompressionSupport.....................................9

SoftwarePackageCompression...................................................10

RemoteDifferentialCompression(Rdc)........................................10

IndependentOutboundEventAndInboundEvent

ProcessingControls.......................................................................11

SelectiveStructureReplication.....................................................11

Rule-BasedItem-LevelContentReplication.................................11

SelectableReplicationEvents.......................................................12

SharedReplicationPackages.......................................................12

PackageDatabaseCaching..........................................................13

ConfigurableReplicationMonitorUpdateLevel...........................13

ImprovedInitialReplicationSupport.............................................14

MICROSOFT TECHNOLOGY CENTER PERFORMANCE

ANDSCALABILITYBENCHMARK...............................................................14

METALOGIX NETWORK TEST LAB PERFORMANCE BENCHMARKING...16

UNDERSTANDING REPLICATOR PERFORMANCE AND SCALABILITY.......................................................................17

MetalogixReplicatorPipeline.......................................17

ReplicatorPerformance...........................................18

OutboundEventProcessing..................................19

PackageTransfer......................................................20

InboundEventProcessing......................................21

Performance Monitoring .........................................22

Replicator Pipeline Performance Features .............24

ReplicatorScalability..............................................25

APPENDIX A – ABOUT WINDOWS REMOTE DIFFERENTIAL COMPRESSION ................................................................ 25

Copyright © 2012 Metalogix International GmbH. All rights reserved. Metalogix is a trademark of Metalogix International GmbH. StoragePoint is a registered trademark of BlueThread Technologies, Inc. Microsoft, Exchange Server, Microsoft Office, SharePoint, and SQL Server are registered trademarks of Microsoft Corporation.

Metalogix Replicatorfor SharePointENTERPRISE PERFORMANCE AND SCALABILITYVersion4.1June2012

CONTENTS

Page 2: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

MetalogixReplicatorisanenterprise-classsolutionforreplicatingcontent,metadata,structure,security,look&feelandbusinessprocessesfromoneSharePointwebapplicationtoantherwebapplication.Replicatorversion4supportsSharePointFoundation2010,SharePointServer2010,OfficeSharePointServer2007andWindowsSharePointServices3.0.

MetalogixReplicatorsupportsthereplicationneedsofMicrosoft’s largest(andsmallest)SharePointcustomers.Version4providesadditionalperformanceandscalabilitycapabilitiesthesecustomerswillneedastheycontinuetogrowtheirDistributedSharePointenvironments.

The release ofMetalogix Replicator version 4 represents a new performance and scalabilitymilestone forSharePointreplicationsolutions.Version4includesnewfeaturesforsupportingmassivelyscalableSharePointreplicationnetworks:

SharedReplicationPackages,

PackageDatabaseCaching,

Improved Initial Replication Support, and

ImprovedRemoteDifferentialCompression(RDC).

Together,thesefeaturesbuildontheperformancefeaturesthathavebeenbuiltintoReplicatorovertheprevious3major releasesand7yearsofdevelopmenteffort.TheReplicatorversion4performance featuresetnowincludesthefollowingcomponents:

1.MultipleReplicationEngineDeployment

2. Multi-threaded Replication Queue Architecture

3.CustomizableReplicationInterval

4.ConfigurablePackageEventCount

5.ConfigurablePackageEventProcessingDuration

6.ReplicationPackageCompression

7.HardwareNetworkCompressionSupport

8.SoftwarePackageCompression

9.RemoteDifferentialCompression(RDC)

10.IndependentControlsforInboundProcessingandOutboundEventProcessing

11.SelectiveStructureReplication

INTRODUCTION

2

Page 3: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

3

BENCHMARKING AT THE MICROSOFT TECHNOLOGY CENTER

12.Rule-basedItem-levelContentReplication

13.SelectableReplicationEvents

14.SharedReplicationPackages

15.PackageDatabaseCaching

16.ConfigurableReplicationMonitorUpdateLevel

Thisdocumentprovidesadditionaldetailed informationon theReplicatorperformance featureset.Thefirstsectiondescribeseachcomponentof theReplicatorperformance featureset.Thesecondsection includesin-depthdiscussionsofeachfeatureaswellasnewbenchmarktestingperformedattheMicrosoftTechnologyCenter inNewYork,NY and theMetalogixNetwork Test Lab. The final section of the document analysesReplicator from an operational point-of-view.

InAugust2010,withthesupportoftheMicrosoftTechnologyCenterinNewYork,NY,theMetalogixMassiveScalabilityTeamexecutedseveralperformanceteststoexerciseapre-releaseversionofMetalogixReplicatorversion4.TheScalabilityTeamconfiguredthelargestknownDistributedSharePoint2010environmentintheworldforthesetests:

61SharePointfarmsconnectedintoasinglereplicationnetwork

Microsoft Hyper-V server virtualization technology

65SharePoint2010Windows2008virtualservers

16Windows2008R2hostphysicalservers

128GBofphysicalRAM

1.2TBofphysicaldiskstorage

Page 4: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

4

TheReplicator version4performance feature set builds uponmany components that havebeenbuilt intoReplicator over the previous 3major releases and 7 years of development effort. The new or significantlyimprovedversion4featuresaremarkedwithanasterisk.

Figure1.MicrosoftTechnologyCenter,NewYorkBenchmark:MetalogixConnectfor Replicator Visualization

 

REPLICATOR PERFORMANCE AND SCALABILITY FEATURES

Apre-releaseversionofMetalogixConnectforReplicatorwasusedtovisualizeandreportonthisconfiguration.Thefullydeployed61SharePointfarmenvironmentisdepictedinFigure1.

Page 5: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

5

Feature DescriptionReplicator Multi-EngineDeployment

ReplicatorEnterpriseEditionenablestheReplicatorEnginetobedeployedonmultiplewebfront-endsineachfarm.Withmultiplewebfront-endshostingReplicatorEngineinafarm,Replicatorprovidesahigheravailabilityandhigherperformancereplicationsolution;inadditiontosupportingincreasedscalability.

Multi-Threaded Replica-tion Queue Architecture

Replicator’sReplicationEnginearchitectureisbasedonamulti-threadedreplica-tionqueuearchitecturethatsupportsparallelReplicationPackageprocessing.

CustomizableReplicationInterval

Replicatorsupportsseveralreplicationschedulingpolicies:Immediately,Interval,Daily,Weekly,MonthlyandManual.

ConfigurablePackageEvent Count

Bydefault,ReplicatorincludestheabilitytogrouporbatchmultipleReplicationEventsintoasingleReplicationPackageforprocessingandtransfertotheTargetWebAp-plication.ThisConfigurablePackageEventCountdeterminesthemaximumnumberofEventsthatcanbepackagedintoasingleReplicationPackage.ThisenablesReplica-tortobetunedfordifferentlevelsofreal-timereplication,wideareanetworkperfor-mancecharacteristicsandavailableInboundandOutboundEventProcessingmemoryand CPU processing resources.

ConfigurablePackageEvent Processing Duration

SimilartotheConfigurablePackageEventCountfeature,theConfigurablePack-age Event Processing Duration allows control over how many Replication Events are groupedorbatchedtogetherintoasingleReplicationPackagebasedonelapsedOutboundEventProcessingtime.ThissettingenablesReplicatorlocalserverresourcerequirementstobetunedforoptimalperformance.

ReplicationPackageCompression

Replicator supports several forms of software compression in addition to support for andcompatibilitywithvarietyofhardwarenetworkcompressiondevices.Replicatorcan use either software or hardware compression to reduce the amount of Replication Packagedatatransferredoverawideareanetwork.

SoftwarePackageCompression

ReplicatorsupportsZIPsoftwarepackagecompressionandacustomimplementationofMicrosoft’sRemoteDifferentialCompression(RDC)thatishighlyoptimizedfortheone-wayandbi-directionalreplicationofSharePointdata.

HardwareNetworkCompression Support

Whenhardwarenetworkcompressiondevices,bandwidthoptimizationappliances,ornetworkacceleratorsolutionsareavailable(suchastheRiverbed®Steelhead®Appli-ance),Replicatorcanbeconfiguredtominimizetheserverresourcesusedforsoftwarecompressionandmaximizetheeffectiveofthenetworkcompressiondeviceorappli-cation.

RemoteDifferentialCompression(RDC)*

Metalogix Replicator supports a custom implementation of Microsoft’s RemoteDifferentialCompression(RDC)technologythatMetalogixhasoptimizedtoreducethetotalamountofReplicationPackagedatathatneedstobetransferredbe-tweentheSourceWebApplicationtotheTargetWebApplicationduringreplication.

IndependentOutboundEventandInboundEventProcessing Controls

ReplicatorsupportsindependentcontrolsforOutboundEventProcessingandIn-boundEventProcessingattheWebApplicationandReplicationConnectionlevelsforaSourceWebApplicationandTargetWebApplication.Inaddition,replicationforaparticularWebApplicationcanbeconfiguredasone-wayorbi-directional.

Page 6: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

6

Feature DescriptionSelective Structure Replication

Bydefault,manycustomersusewebapplication-to-webapplicationreplication – replicating the entire content of each site collection inonewebapplicationtothesecondwebapplication.SelectivestructurereplicationenablestheSharePointAdministratortoselectaspecificsubsetoftheSourceWebApplicationstructuretobereplicatedtotheTargetWebApplication.

Rule-basedItem-levelContent Replication

Rule-baseditem-levelcontentreplicationusestheMetalogixReplicatorRulesEngineto process custom rule sets to determine if an item in a particular list or document libraryshouldbereplicated.

SelectableReplicationEvents

ReplicatorsupportsselectablereplicationofchangesthatoccurinaSharePointWebApplication,SiteCollection,WebSite,ListorDocumentLibrary.Thedifferenttypesofindividual changes are called Replication Events. Replication Events are categorized intohigher-levelEventGroups.ByselectingwhichEventsneedtobereplicatedandwhichEventsdon’tneedtobereplicated,theSelectableReplicationEventsfeatureprovidestheSharePointAdministratorwithfine-grainedcontrolovertheEventsprocessingduringInboundEventProcessing,PackageTransferandOutboundEventProcessing; which in turn help improve overall Replicator performance.

Shared Replication Packages*

InReplicatorversion3,aseparateReplicationPackagewascreatedforeachoutboundReplicationConnection.TheSharedReplicationPackagefeatureinReplicatorversion4eliminatesthetimerequiredtocreateaseparatePackageforeachoutboundReplicationConnection–asinglesharedPackagecontainingthebatchofReplicationEventsiscreated.DependingonthenumberofEventsinthepackage,thesizeandtypeoftheSharePointchange,andthenumberofoutboundReplicationCon-nections,thiscansignificantlyreduceOutboundEventProcessingtimeandresources.

PackageDatabaseCaching*

InReplicatorversion4,uponacceptanceofthetransferofanInboundReplicationPackage,ReplicatorimmediatelycachesthemetadataforeachReplicationEventintheReplicatorConfigurationdatabase.ThisimprovestheperformanceoftheReplica-torEnginebyeliminatingtheneedtorepeatedlyaccesstheserializedPackagedata;especially in SharePoint farms that host multiple Replicator Engines.

ConfigurableReplicationMonitorUpdateLevel*

DuringnormalInboundEventProcessing,Replicator,runningonaTargetWebApplication,willmakeseveralcallstotheReplicatorWebServicerunningontheSourceWebApplicationtokeeptheMonitorReplicationreportsontheSourceWebApplicationascurrentaspossible.TohelpminimizetheQueuedItemupdatenet-worktrafficbetweentheSourceandTargetWebApplications,Replicatorversion4.1supportsanewReplicationConnectionpropertycalled“quietmode”.

Improved Initial ReplicationSupport*

ThenewBackupModefeatureprovidesamoreefficientmethodofreplicatingalargesitecollectionorhierarchyofwebsites.UsingBackupMode,ReplicatorusestheSharePoint import and export operations to create a single archive of the entire group ofwebsites,andqueuesthereplicationofthisarchiveasasingleReplicationEvent.

*NeworsignificantlyimprovedReplicatorversion4feature

Page 7: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

7

ReplicatorEnterpriseEditionenablestheReplicatorEnginetobedeployedonmultiplewebfront-endsineachfarm.Withmultiplewebfront-endshostingReplicatorEngineinafarm,Replicatorprovidesahigheravailabilityandhigherperformance replicationsolution; inaddition tosupporting increasedscalability. Ifoneengine isstopped, the other engines are still operational.

Thefollowingisbasedonaversion4benchmarkperformedattheMicrosoftTechnologyCenterinNewYorkusingahub-and-spoketopologyconnecting61SharePoint2010farms.Thefigureshows:

Multiplereplicationenginesprocessingandtransferringmultiplepackagessimultaneously

MultiplethreadsperenginesupportingparallelReplicationPackageprocessingandtransfer

Smooth scaling of the replication processing under heavy load

Replicator’sReplicationEnginearchitectureisbasedonamulti-threadedreplicationqueuearchitecturethat supports parallelReplicationPackageprocessing.ReplicatorStandardEdition supports 4 threadsperreplicationenginewithasingleactiveenginerunningonasinglewebfront-endperSharePointfarm.ReplicatorEnterpriseEditionsupports10toanunlimitednumberofthreadsrunningonasinglereplicationenginerunningonanunlimitednumberofwebfront-endsperSharePointfarm.

Figure2.MultipleEngine,MultipleThreadPerformanceandScalability showsReplicatorEnterpriseEditionrunningon4webfront-endsconfiguredtoperformparallelReplicationPackageprocessingandtransfer.

Figure2.MultipleEngine,MultipleThreadPerformanceandScalability

REPLICATOR MULTI-ENGINE DEPLOYMENT

MULTI-THREADED REPLICATION QUEUE ARCHITECTURE

 

Page 8: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

8

CUSTOMIZED REPLICATION INTERVAL

CONFIGURABLE PACKAGE EVENT COUNT

Replicatorsupportsseveraldifferentreplicationschedulingpolicies:

Immediately(defaultreal-time,event-drivenreplicationpolicy)

Interval(seconds,minutes,hours)

Daily

Weekly

Monthly

Manual

Bydefault,ReplicatorincludestheabilitytogrouporbatchmultipleReplicationEventsintoasingleReplicationPackageforprocessingandtransfertotheTargetWebApplication.ThisConfigurablePackageEventCountdetermines themaximumnumberofEvents that canbepackaged into a singleReplicationPackage. ThisenablesReplicator to be tuned for different levels of real-time replication,wide area network performancecharacteristicsandavailableInboundandOutboundEventProcessingmemoryandCPUprocessingresources.

AhigherPackageEventCountwillresultinfewer,largerPackagesbeingcreatedwhichcanincreaseoverallperformance.TheWindowsBackground IntelligentTransferProcess (BITS)protocol usedbyReplicator forPackageTransferisdesignedforhighperformanceandreliability(restartable)withlargefiledownloadsaswellassmallerdownloads.Largerpackages,somewhatsurprisingly,canresultinbetterthroughputandperformanceinslowandunreliablewideareanetworkscenarios.

ThesesettingsenablefinegrainedcontroloverwhenOutboundEventProcessingandPackageTransferwilloccur.This in turnsallows theSharePointAdministrator tobettermanageSharePointserverandwideareanetworkresources.

Figure3.MapFamilyReplicationSchedule

 

Page 9: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

9

SimilartotheConfigurablePackageEventCountfeature,theConfigurablePackageEventProcessingDurationallowscontroloverhowmanyReplicationEventsaregroupedorbatchedtogether intoasingleReplicationPackage based on elapsedOutbound Event Processing time. This setting enables Replicator local serverresourcerequirementstobetunedforoptimalperformance.

Replicatorsupportsseveral formsofsoftwarecompression inadditiontosupportforandcompatibilitywithvarietyofhardwarenetworkcompressiondevices.ReplicatorcanuseeithersoftwareorhardwarecompressiontoreducetheamountofReplicationPackagedatatransferredoverawideareanetwork.

ThefollowingsectionsdescribethebenefitsofReplicator’ssupportforhardwarenetworkcompressionandsoftwarepackagecompression.Replicator’shighlyoptimizedimplementationofMicrosoft’sRemoteDifferentialCompression(RDC)isalsodescribed.

When hardware network compression devices, bandwidth optimization appliances, or network acceleratorsolutions are available (such as the Riverbed® Steelhead® Appliance), Replicator can be configured tominimizetheserverresourcesusedforsoftwarecompressionandmaximizetheeffectivenessofthenetworkcompression device or application. A sample deployment is illustrated in Figure 4. Metalogix Replicator and Riverbed®Steelhead®ApplianceCompressedReplicationPackageSolution.

Figure4.MetalogixReplicatorandRiverbed®Steelhead®ApplianceCompressedReplicationPackageSolution

CONFIGURABLE PACKAGE EVENT PROCESSING DURATION

REPLICATION PACKAGE COMPRESSION

HARDWARE NETWORK COMPRESSION SUPPORT

BranchOffice

LAN

WAN

Metalogix ReplicatorData Center

Metalogix Replicator on RSP

Page 10: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

10

ConfiguringReplicatortosupporthardwarenetworkcompressiondevicesavoidstheredundantandpotentiallyexpensiveoverheadofcompressingpackagesinsoftwarewhenhardwarecompressionisavailable.

SOFTWARE PACKAGE COMPRESSION

REMOTEDIFFERENTIALCOMPRESSION(RDC)

ReplicatorsupportsZIPsoftwarepackagecompressionandacustomimplementationofMicrosoft’sRemoteDifferential Compression (RDC) that is highly optimized for the one-way and bi-directional replication ofSharePoint data.

MetalogixReplicatorsupportsacustomimplementationofMicrosoft’sRemoteDifferentialCompression(RDC)technologythatMetalogixhasoptimizedtoreducethetotalamountofReplicationPackagedatathatneedstobetransferredbetweentheSourceWebApplicationtotheTargetWebApplicationduringreplication.

“Remote Differential Compression (RDC) allows data to be synchronized with a remote sourceusingcompressiontechniquestominimizetheamountofdatasentacrossthenetwork.RDCissuitableforapplicationsthatmovedataacrossawideareanetwork(WAN)wherethedatatransmissioncostsoutweightheCPUcostofsignaturecomputation.RDCcanalsobeusedonfasternetworksiftheamountofdatatobetransferredisrelativelylargeandthechangestothedataaretypicallysmall.”1MorespecificdetailsonRDCcanbefoundinAppendixA–AboutWindowsRemoteDifferentialCompression.

Metalogix’sDynamicDifferentialCacheimprovementstoMicrosoftRDCuseserver-sidepre-processingandpost-processingtosignificantlyreducetheamountofSharePointdatathatistransferredoverthewideareanetwork.

DuringperformancetestingintheMetalogixNetworkTestLab,usingreal-lifeandsimulatedtestdata(SharePointtasks,multi-leveldocumentlibraries,Office2003documents,Office2007documents,Office2010documentsaswellasotherdesktopfileformatssuchasAutoCADdrawings,photosandimages),MetalogixReplicatorRDCreducedthetotalnetworktrafficbymorethan90%comparedtoZIPcompression.

Page 11: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

INDEPENDENT OUTBOUND EVENT AND INBOUND EVENT PROCESSING CONTROLS

SELECTIVE STRUCTURE REPLICATION

RULE-BASED ITEM-LEVEL CONTENT REPLICATION

ReplicatorsupportsindependentcontrolsforOutboundEventProcessingandInboundEventProcessingattheWebApplicationandReplicationConnectionlevelsforaSourceWebApplicationandTargetWebApplication.Inaddition,replicationforaparticularWebApplicationcanbeconfiguredasone-wayorbi-directional.

Whenonlyone-wayreplication isrequired,performance isoptimizedwhenEventprocessing isdisabledontheappropriateWebApplication.Inthebi-directionalreplicationscenario,separateReplicationConnectionsenableseparatereplicationpoliciestobeusedfortheoutboundandinboundconnections.

Bydefault,manycustomersusewebapplication-to-webapplicationreplication–replicatingtheentirecontentofeachsitecollection inonewebapplicationtothesecondwebapplication.SelectivestructurereplicationenablestheSharePointAdministratortoselectaspecificsubsetoftheSourceWebApplicationstructuretobereplicatedtotheTargetWebApplication.

Forexample:

One or more site collections

Rootwebsite

Anyindividualchildwebsite

Anychildwebsiteandsubhierarchyofthechildwebsite’schildwebsites

Individuallistsand/ordocumentlibrariesinawebsite.

Selectivestructurereplicationcanbeusedtosignificantlyreducethevolumeofreplicationdatabyselectingthespecificsitecollectionsandwebsitestobereplicated.

Rule-based item-levelcontent replicationuses theMetalogixReplicatorRulesEngine tooptionallyprocesscustomrulesetstodetermineifaniteminaparticularlistordocumentlibraryshouldbereplicated.

Althoughrules-basedcontentreplicationisnormallyconsideredasecurityandcontentpromotionfeature,rulesetscanalsobeusedtoblockthereplicationoflargerfilesorotherfilesbasedoneachitem’smetadatavaluesand improve performance.

11

1Source:http://msdn.microsoft.com/en-us/library/aa372948(v=VS.85).aspx)

Page 12: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

SELECTABLE REPLICATION EVENTS

SHARED REPLICATION PACKAGES

12

Replicator supports selectable replication of changes that occur in a SharePoint Web Application, SiteCollection,WebSite,ListorDocumentLibrary.ThedifferenttypesofindividualchangesarecalledReplicationEvents. Replication Events are categorized into higher-level Event Groups. By selecting which Events need tobereplicatedandwhichEventsdon’tneedtobereplicated,theSelectableReplicationEventsfeature provides the SharePoint Administrator with fine-grained control over the Events processing duringInboundEventProcessing, Package Transfer andOutboundEventProcessing;which in turn help improveoverall Replicator performance.

Figure5.MetalogixReplicatorReplicationEventshighlightstheReplicationEventssectionof theConfigureMapFamilywebpagefromReplicatorCentralAdministration.EntireEventGroupsorindividualEventscanbeenabledordisabled.Thisfigureshowsthedefaultsettings.

InReplicatorversion3,aseparateReplicationPackagewascreatedforeachoutboundReplicationConnection.The Shared Replication Package feature in Replicator version 4 eliminates the time required to create aseparatePackageforeachoutboundReplicationConnection–asinglesharedPackagecontainingthebatchofReplicationEvents iscreated.Dependingon thenumberofEvents in thepackage, thesizeand typeoftheSharePointchange,andthenumberofoutboundReplicationConnections, thiscansignificantly reduceOutboundEventProcessingtimeandresources.

Figure6.MicrosoftTechnologyCenterBenchmark:SharedReplicationPackageOutboundProcessingTimeillustrateshowtheelapsedOutboundProcessingtimeforthefirstPackage(greenportionoffirstbarinthechart)wassignificantlyreducedfortheremaining59instancesofthePackagereplicatedtoTargetWebApplication2through60.

Replication version4.1 also includesnewWebSiteReplicationEvents for replicating SharePoint 2010’ssocialnetworkingfeatures:ratings,commentsandkeywordtags.

Figure 5. Metalogix Replicator Replication Events

 

Page 13: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

PACKAGE DATABASE CACHING

CONFIGURABLE REPLICATION MONITOR UPDATE LEVEL

In Replicator version 4, upon acceptance of the transfer of an Inbound Replication Package, ReplicatorimmediatelycachesthemetadataforeachReplicationEvent in theReplicatorConfigurationdatabase.ThisimprovestheperformanceoftheReplicatorEnginebyeliminatingtheneedtorepeatedlyaccesstheserializedPackagedata;especiallyinSharePointfarmsthathostmultipleReplicatorEngines.

PackageDatabaseCachingspecificallyappliestoInboundEventProcessingontheTargetWebApplicationServerandimprovesoverallInboundEventProcessingbyreducingthenumberoffilesystemaccesses,reducingthevolumeofdatareadfromthefilesystemandloweredfiledecompressionCPUusage.

DuringnormalInboundEventProcessing,Replicator,runningonaTargetWebApplication,willmakeseveralcallstotheReplicatorWebServicerunningontheSourceWebApplicationtokeeptheMonitorReplicationreportsontheSourceWebApplicationascurrentaspossible.

TohelpminimizetheQueuedItemupdatenetworktrafficbetweentheSourceandTargetWebApplications,Replicatorversion4.1supportsanewReplicationConnectionpropertycalled“quietmode”.Itcanbeconfigured through theConfigureReplicationConnectionadministrationpagewhere it is referred toas theConfigurableReplicationMonitorUpdate Level. This new setting has 2 values:Normal andMinimal (quitemode).

DuringperformancetestingintheMetalogixNetworkTestLabreplicating10001KBfiles,thenetworktrafficwasreducedby55%andthenumberofHTTPPUToperationswasreducedby60%.

Figure6.MicrosoftTechnologyCenterBenchmark:SharedReplicationPackageOutboundProcessingTime

13

 

Page 14: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

ThenewBackupModefeatureprovidesamoreefficientmethodofreplicatingalargesitecollectionorhierarchyofwebsites.UsingBackupMode,ReplicatorusestheSharePointimportandexportoperationstocreateasinglearchiveoftheentiregroupofwebsites,andqueuesthereplicationofthisarchiveasasingleReplicationEvent.

BackupMode significantly improves the performance and reliability of a large initial replication: instead ofcreatingandtransferring10,000sofqueueditemsandpackagesinaverylargeSharePointenvironment,onlyafew(large)Packagesneedtobepackagedandtransferred.

Inaddition,ifRDCcompressionisenabled,theamountofdatatransferredoverawideareanetworkwillbereduceddramatically.Whenreplicatingtomultiplelocations,onlyonePackageiscreatedforalldestinationswiththeversion4SharedReplicationPackagefeature.

Figure 7. Microsoft Technology Center Windows Server 2008 R2 Hyper-V Host Environment: MetalogixReplicatorBenchmark

InAugust2010,MetalogixwasinvitedtotheMicrosoftTechnologyCenter(MTC)inNewYork,NYtomeasuretheperformanceandscalabilityofanearlyreleaseofMetalogixReplicatorforSharePointversion4.0.

WorkingcloselywiththeMTCtechnicalstaff,theMetalogixMassiveScalabilityTeamdeployedandconfiguredtheworld’slargestSharePoint2010DistributedSharePointenvironment,measuredintermsofthenumberof independent SharePoint farms. Eleven large scale physical servers from Dell and HP running Microsoft WindowsServer2008R2wereusedtodeployandconfigure85WindowsServer2008

R2 virtual machines. Microsoft Windows Hyper-V was used for the operating system virtualization solution. The entireconfigurationwasdeployedandmanagedusingMicrosoftWindowsVirtualMachineManager.

IMPROVED INITIAL REPLICATION SUPPORT

MICROSOFT TECHNOLOGY CENTER PERFORMANCE AND SCALABILITY BENCHMARK

14

HUB Servers

64GB

x 16

Core

s

Hype

r-V H

osts

3 x 200GB LUNs 200GB LUN 200GB LUN 200GB LUN

x 8

32GB

x 16

Core

s

32GB

x 16

Core

s

32GB

x 16

Core

s

VPN VMM RouterAD/DNS LabWorkstations

TARGET Servers TARGET Servers TARGET Servers

Page 15: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

15

Fivevirtualmachinesrunningon16processorcoreswithaccessto64GBofRAMand600GBofdiskstoragewereusedtohost4WebFront-EndServersand1SQLServer2008databaseserverintheHubSharePoint2010farm.Eighty(80)additionalvirtualmachinesrunningon96processorcoreswithaccessto320GMofRAMand2TBofdiskstoragewereusedtohost60single-serverSharePoint2010farms.ThespecificconfigurationoftheWindowsServer2008R2physicalhostscanbefoundinTable1.MicrosoftTechnologyCenterWindowsServer2008R2Hyper-VHostEnvironment:VirtualMachineConfiguration.

Table1.MicrosoftTechnologyCenterWindowsServer2008R2Hyper-VHostEnvironment:VirtualMachineConfiguration

Thefollowingperformancemonitoringtoolswereusedduringthebenchmark:

Windows Performance Monitor

Metalogix Replicator Replication Monitor

Metalogix Replicator Replication Status

Metalogix Connect for Replicator

Custom Replication Data Collection PowerShell Scripts

CustomReplicatorDeployment,Configuration,andManagementPowerShellScripts

CustomSharePointBenchmarkingPowerShellScripts

HostType

No.Hosts

Cores RAM(GB)

No.LUN

LUNSize(GB)

TotalCores

TotalRAM

TotalStorage(GB)

VMRAM

No. VM

Hub 1 16 64 3 200 16 64 600 - 5

Targets 1 16 32 1 200 16 32 200 4 8

Targets 1 16 32 1 200 16 32 200 4 8

Targets 8 8 32 8 200 64 256 1600 4 64

Totals 11 112 384 2600 12 85

Page 16: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

DuringthedesignanddevelopmentofMetalogixReplicatorversion4,eachnewperformancebuildwastestedandmeasuredalong40dimensions,including:

CPU usage

Availablememory

Workingset

Diskaccess

Networkaccess

Networkutilization

HTTPWebServerperformance

Theconfigurationof theMetalogixNetworkTest Lab is depicted in Figure 8.MetalogixNetworkTest Lab:PerformanceBenchmarking.IncontrastwithvirtualizedMicrosoftTechnologyCenterenvironmentusedforthemassivescalabilityandperformancebenchmarking,theMetalogixNetworkTestLabrunsentirelyonphysicalhardware.

Figure8.MetalogixNetworkTestLab:PerformanceBenchmarking

METALOGIX NETWORK TEST LAB PERFORMANCE BENCHMARKING

16

RDC1 SQL Server

Phys

ical H

osts

RDC1 WFE

WAN Simulator

AD/DNS Lab Workstations

RDC2 WFE RDC2 SQL Server

Page 17: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

ThekeytounderstandingReplicatorperformanceandscalability fromaday-to-dayoperationalperspectiveinvolvesunderstandinghowthebasiccomponentsoftheReplicatorPipelinecaptureandpackageReplicationEvents,transferpackagesfromtheSourceWebApplicationtotheTargetWebApplicationandfinallyupdatetheTargetWebApplicationwiththeneworchangedcontent.

OutboundEventProcessing is responsible forcapturingandrecordingReplicationEvents thatoccur in theSourceWebApplication.Forexample,whenausercreatesanewcontactinacontactlistinaSharePointWebsiteoreditsandsavesaMicrosoftWorddocumentthatisstoredinaSharePointdocumentlibrary,eachoftheseis captured and recorded as a Replication Event. Replication Events are captured and stored in a set of SQL ServertablesinaReplicatordatabase.OutboundEventProcessingiscontrolledbyReplicationMapswhichdeterminewhatEventsneedtobecaptured,packagedandtransferredtotheTargetWebApplication.GroupsofReplicationEventsarepackaged into two typesofmessagesorobjects:Queued ItemsandReplicationPackages.

Figure9.MetalogixReplicatorPipeline

The Replicator Pipeline is a detailed representation of how Replicator divides the overall replication process into 3activities:

CapturesachangeonaSourceWebApplication,packagesagroupofReplicationEventsandassociatedSharePointcontentintoaQueuedItemandcorrespondingReplicationPackage,

TransferstheQueuedItemandPackagefromtheSourcetotheTargetWebApplication,and

WhenreceivedandacceptedontheTargetWebApplication,appliesthechangesrepresentedbytheQueuedItemandPackagetotheTargetWebApplication.

These3activities,OutboundEventProcessing,PackageTransferandInboundEventProcessingareshowninthefollowingfigure.

UNDERSTANDING REPLICATOR PERFORMANCE AND SCALABILITY

METALOGIX REPLICATOR PIPELINE

 

17

Page 18: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

AQueued Item is a unit ofwork to be transferred to a TargetWebApplication for remote execution. TheReplicatorWebServiceonaTargetWebApplication iscalledtopushaQueuedItemfromtheSourceWebApplicationtoaTargetWebApplication.

A Replication Package is a collection of one or more Replication Events plus data about the SharePointinformation that is packaged in a format specific to theReplication Transport being used.When anEventisbeingprocessed,OutboundEventProcessingcalls theSharePointobjectmodel toextract thechangedSharePointinformationfromtheSourceWebApplicationcontentdatabase.

The Package Transfer activity is responsible for the transfer of Queued Items and Replication Packages(Packages)fromtheSourceWebApplicationtotheTargetWebApplication.PackageTransferistheprocessthatsitsbetweenOutboundEventProcessing(ontheSourceWebApplication)andInboundEventProcessing(ontheTargetWebApplications).PackageTransferinvokesReplicatorEnginesoftwarecomponentsonboththeSourceandTargetWebApplications.

InboundEventProcessingisresponsibleforprocessingtheQueuedItemsandPackagesreceivedandacceptedbytheTargetWebApplicationandapplyingthemtothesitecollections,Websites,listsanddocumentlibrariesinthewebapplication’scontentdatabase.TheQueuedItemsandPackagesareappliedtotheTargetWebApplicationcontentbasebycallingtheSharePointObjectModel.

ReplicatorperformanceisbestunderstoodbyanalyzingtheReplicatorPipelineactivitiesdepictedinFigure9.MetalogixReplicatorPipeline.ThesamevariablesthataffectperformanceofaSharePointenvironmentalsoaffectReplicatorperformance.ReplicatorwillperformoptimallyiftheunderlyingSharePointenvironmenthasbeenconfiguredforoptimalperformanceforagivenuserworkload.Inaddition,thereareReplicator-specificconfigurationsettingsandoptimizationthatcanfurtherimproveReplicatorperformance.

TheperformanceprofileofSharePoint,andeachoftheReplicatorPipelineactivities,isbasedonavarietyoffactors including:userworkload,documentfile formats,documentsizedistributions,SharePointserverandfarmconfiguration,databaseserverconfiguration,localareanetwork(LAN)characteristics,wideareanetwork(WAN)characteristics,ReplicationdatacompressionandreplicationmonitorsettingsaswellasotherReplicatorconfigurationsettings.

Userworkloadisdefinedbytwofactors:usercontentchangerateandusercontentchangesizedistribution.Qualitatively, the user content change rate is simply the rate at which users of a particular SharePoint environment makecontentchanges:creatingnewlistitems,editingandsavingexisting

items, creating or uploading new documents, editing and saving existing documents and, less frequently,

REPLICATOR PERFORMANCE

18

Page 19: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

creatingnewsitecollections,websites,listsanddocumentlibrariesormakingmetadataorsecuritychanges.

User content change size distribution refers to the size of the content changes and their distribution. Forexample,inadocumentcollaborationsolution,theaveragesizeofaWorddocumentmightbeafewmegabytesandtheaveragesizeofaPowerPointmightbemeasuredin10’sofmegabytes.TheamountofdataneededtorepresentanewSharePointcontactortaskitemwouldbemeasuredin10sor100sofkilobytes.Thedistributionofsizesvaryfromsmall1Kbytedocumentstomediafilesmeasuredin10sofmegabytes.

In Replicator version 4, each Map Family contains a Replication Schedule. A Replication Schedule controls how frequentlyReplicationEventsforaparticularReplicationConnectionareprocessedandpackagedintoQueuedItemsandReplicationPackages.EventProcessingandPackagingcanbescheduledtorun:

Immediately(real-timereplication)

Afteratimedinterval(afteraspecifiednumberofseconds,minutes,hoursordays)

Scheduleddaily,weekly,monthly

Manually initiated

Withtheexceptionofimmediatereplication,OutboundEventProcessingperformanceisprimarilydeterminedbytheReplicationSchedule,usercontentsizedistributionandSharePointsystemperformance.Forimmediatereplication,OutboundEventProcessingperformanceisalsoaffectedbytheusercontentchangerate.Fromasystemperformanceperspective,allOutboundEventProcessingoccurslocallyintheSharePointfarmthatishostingaparticularSourceWebApplication.OnthelocalSharePointfarm,serverRAM,diskorganization,LANnetworkconfigurationanddatabaseserverconfigurationarethegreatestdeterminantsofSharePointandReplicator system performance.

When Replicator is configured for immediate replication, Outbound Event Processing (Event capture andprocessing)isperformedinnearrealtime.

OUTBOUND EVENT PROCESSING

19

Page 20: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

PackageTransferreferstothepartoftheReplicatorPipelinethatisresponsiblefortransferringQueuedItems(and their correspondingPackages) fromaSourceWebApplication to a TargetWebApplication. PackageTransferperformanceisdeterminedbyanumberofdifferentfactors:

Userworkload

WANnetworkperformance

SourceWebApplicationsystemperformance

TargetWebApplicationsystemperformance

Replicator data compression

AQueuedItemandcorrespondingReplicationPackagerepresentsabatchofchangesthatusershavemadetotheSourceWebApplicationcontent.QueuedItemsarerelativelysmallinsizeandarepassedfromtheSourceWebApplicationtotheTargetWebApplicationbycallingtheTargetWebApplicationReplicatorWebService.PackagesareeitherdownloadedoruploadedtotheTargetWebApplicationusingtheBITSprotocolrunningoverHTTP/HTTPS.ThemajorityofthePackageTransfertimeisconsumedbythetransferofthePackage.ThisisbecausethePackagecontainsthatchangedSharePointcontentthatneedstobereplicatedfromtheSourceWebApplicationtotheTargetWebApplicationandthechangedcontent,inrelativeterms,maybequitelargedependingonwhetherthechangedcontentissimplelistitemora10MBPowerPointdocument.

Package Transfer performance be improved using Package compression settings that can be set usingReplicatorCentralAdministrationonaReplicationConnectionbyConnectionbasis.The followingPackagecompressionsettingsaresupportedinReplicator:

None

ZIP(default)

RDC(basedonMicrosoftRemoteDifferentialCompressiontechnology)

OncethecharacteristicsoftheQueuedItemandPackagestreamandthePackagecompressionsettingsareunderstood,thenextmajordeterminantofPackageTransferperformancearethecharacteristicsoftheWAN:

Availablebandwidth

Round-trip latency

Packetloss

Reliability

For example, satellite links typically have lower bandwidth, high latency and lower reliability compared toterrestriallinks;andexceptionallypoorerperformancecomparedtoafiberringinametropolitanareanetwork(MAN) or LAN.Metalogix’s advanced use of theMicrosoft BITS protocol in the design of theReplicatorPackageTransferprocessallowsReplicatortofunctionextremelywelloverhighperformanceconnectionsaswellasoversloworunreliablesatellite,ship-to-shoreandbattlefieldnetworks.

PACKAGE TRANSFER

20

Page 21: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

IfReplicatorisinstalledandconfiguredonmultipleSharePointfarmsconnectedbyahigh-performancenetwork,thePackageTransferprocesswillperformonanearrealtimebasisbetweenanypairofSourceandTargetWebApplications.

InboundEventProcessingbeginsonceaQueuedItemanditscorrespondingPackagehavebeentransferredtotheTargetWebApplicationbythePackageTransferprocess.InboundEventProcessingexecutesentirelywithintheSharePointfarmfortheTargetWebApplication.

InboundEventProcessingperformanceisdeterminedbytherateatwhichtheReplicatorServicecanprocessthelocalQueuedItemsandtheircorrespondingPackages,decompressthePackagesandapplythecontentchangestotheTargetWebApplicationbycallingtheSharePointObjectModel.TheInboundEventProcessingperformanceisaffectedbythefollowingfactors:

Userworkload

SharePointserverandfarmconfiguration

Databaseserverconfiguration,

Localareanetwork(LAN)characteristics

Subject to theaboveconstraints,Replicator InboundEventProcessing isable tooperateatnear real timeperformance.

INBOUND EVENT PROCESSING

21

Page 22: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

Replicator Central Administration provides detailed replication monitoring, replication status and replication eventprocessinghistoryreportsformonitoringtheReplicatorPipeline.ThefollowingtwofiguresareexamplesofthestatisticsavailablefromtheReplicationStatusreportforasampleintranetSourceWebApplicationthatisbeingreplicatedtoanextranetTargetWebApplication.

NOTE:Thefollowingfiguresarefromasmall-scaletestperformedonvirtualsingleserverSharePointfarmthatishostingboththeSourceandTargetWebApplications.Thestatisticsshowninthesefiguresareforillustrationpurposes only.

Intheabovefigure,theOutboundEventProcessingstatisticstellsus:

13Packages(andQueuedItems)wereprocessedrepresenting13batchesofReplicationEvents

TotalnumberofReplicationEvents=212

Totalsizeofthe13Packages(212Events)=13.8MB

Figure10.SampleOutboundEventProcessingStatistics

PERFORMANCE MONITORING

 

22

Page 23: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R  

Figure11.SampleInboundEventProcessingStatistics

Intheabovefigure,theInboundEventProcessingstatisticstellsus:

14Packages(andQueuedItems)wereprocessedrepresenting14batchesofReplicationEvents

TotalnumberofReplicationEvents=213

Totalsizeofthe14Packages(213Events)=11.82MB

NOTE:InboundandOutboundEventProcessingstatisticsareneverexpectedtomatch,duetoseveralfactors,includingsystemgeneratedQueuedItemsandPackages,retries,filteredPackagesandrejectedPackages.

TomonitorlargeDistributedSharePointenvironmentsconsistingofseveralfarms,MetalogixConnectfor

SharePoint or Metalogix Connect for Replicator is the monitoring tool of choice for these environments.

23

Page 24: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

ThefollowingtablemapsthekeyMetalogixReplicatorperformancefeaturestoeachphaseoftheReplicatorPipeline.

NOTE:TheperformancefeaturesthataffectOutboundEventProcessingalsotendtoenhancetheperformanceofthedownstreamReplicatorPipelineactivities:PackageTransferandInboundEventProcessing.Thereisonlyonefeature,PackageDatabaseCaching,whichonlyaffectsInboundEventProcessing.

Table2.MetalogixReplicatorPipelinePerformanceFeatures

REPLICATOR PIPELINE PERFORMANCE FEATURES

24

Feature AllPipelineActivities

OutboundProcessing

PackageTransfer

InboundProcessing

Replicator Multi-Engine Deployment YES

Multi-Threaded Replication Queue Architecture YES

ConfigurablePackageEventCount YES

ConfigurablePackageEventProcessingDuration

YES

IndependentOutboundEventProcessingandIn-boundEventProcessingControls

YES

Selective Structure Replication YES

Rule-basedItem-levelContentReplication YES

SelectableReplicationEvents YES

CustomizableReplicationInterval YES

SharedReplicationPackages YES

ReplicationPackageCompression YES YES

HardwareNetworkCompressionSupport YES YES

SoftwarePackageCompression YES YES

RemoteDifferentialCompression(RDC) YES YES

PackageDatabaseCaching YES

Page 25: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

TheMetalogixReplicatorarchitectureisbasedonanunlimitedmulti-master,point-to-pointreplicationnetworkmodelthatdoesnotrequireglobalsharingofanyconfigurationinformation.Arbitrarynetworktopologiesaresupportedincluding:

Serial

Parallel

Hub-and-spoke

Cascadinghub

Federatedhub

Cross domain

Mesh, and

Offline.

MetalogixConnect isadesktopapplication thatprovidesvisualization,monitoringandmanagementof thewide area SharePoint environment. It can be used to visualize the status of each farm and its associatedconnections.Connectisavailableasanadd-onfeaturefortheBasicandtheStandardEdition(andisincludedintheEnterpriseEdition).

MetalogixConnectforReplicatorprovidesthehighlyscalablereplicationnetworkmanagementtoolsneededtomonitorandmaintainthelargest(andsmallest)distributedSharePointenvironments.

Source:http://msdn.microsoft.com/en-us/library/aa372948(VS.85).aspx

RemoteDifferentialCompression(RDC)allowsdatatobesynchronizedwitharemotesourceusingcompressiontechniquestominimizetheamountofdatasentacrossthenetwork.

RDCisdifferentfrompatching-orienteddifferencingmechanisms,suchasBinaryDeltaCompression(BDC),thataredesignedtooperateonlyonknownversionsofasinglefile.BDCrequirestheservertohavecopiesofallversionsofthefile,anddifferencesbetweeneachpairofversionsarepre-computedsothattheycanbedistributedefficientlyfromaservertomultipleclients.

REPLICATOR SCALABILITY

UNLIMITED MULTI-MASTER, POINT-TO-POINT REPLICATION NETWORK MODEL

METALOGIX CONNECT FOR REPLICATOR

APPENDIX A – ABOUT WINDOWS REMOTE DIFFERENTIAL COMPRESSION

25

Page 26: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

RDCmakesnoassumptionsaboutfilesimilarityorversioning.Becausedifferencesbetweenfilesarecomputedonthefly,RDCisideallysuitedforsynchronizingfilesthataredifferentorhavebeenupdatedindependently.

RDC does not assume that the file data to be synchronized resides in physical files. Therefore, the RDCapplicationisresponsibleforperformingfileI/OonbehalfoftheRDClibrary.

Becauseitistransportindependent,RDCcanbeusedwithRPC,HTTP,orotherdesiredtransportmechanisms.TheRDCapplicationbearstheresponsibilityforchoosingtheappropriatetransportandperforminganyclientor server authentication that is required to support the transport's security model.

RDCissuitableforapplicationsthatmovedataacrossawideareanetwork(WAN)wherethedatatransmissioncostsoutweightheCPUcostofsignaturecomputation.RDCcanalsobeusedonfasternetworksiftheamountofdatatobetransferredisrelativelylargeandthechangestothedataaretypicallysmall.

RDCdividesafile'sdataintochunksbycomputingthelocalmaximaofafingerprintingfunctionthatis

computedateverybytepositioninthefile.Afingerprintingfunctionisahashfunctionthatcanbecomputedincrementally.Forexample,ifyoucomputethefunctionFoverarangeofbytesfromthefile,Bi...Bj,itshouldthenbepossibletocomputeF(Bi+1...Bj+1)incrementallybyaddingthebyteBj+1andsubtractingthebyteBi.Therangeofbytesfromthefile,Bi...Bj,iscalledthehashwindow.Thelengthofthiswindow,inbytes,iscalledthe hash window size.

TheRDClibrary'sFilterMaxsignaturegenerator"slides"thehashwindowacrosstheentirefilebyaddingthebyteattheleadingedgeandsubtractingthebyteatthetrailingedgeofthewindow.Meanwhile,thegeneratorcontinuallyexaminesthesequenceoffingerprintfunctionvaluesoveragivenrangeofbytes,calledthehorizonsize.Ifafingerprintfunctionvalueisalocalmaximumwithintherange,itsbytepositionischosenasa"cutpoint,"orchunkboundary.

After thefilehasbeendivided intochunks, thesignaturegeneratorcomputesastronghashvalue (anMD4hash),calledasignature,foreachchunk.Thesignaturescanbeusedtocomparethecontentsoftwoarbitrarilydifferentversionsofafile.

Becausethesizeofthesignaturefilegrowslinearlywiththesizeoftheoriginalfile,comparingverylargefilescanbeexpensive.ThiscostisreduceddramaticallybyapplyingtheRDCalgorithmrecursivelytothesignaturefiles.Forexample,iftheoriginalfilesizeis9GB,thesignaturefilesizewouldtypicallybeabout

81MB.IftheRDCalgorithmisappliedtothesignaturefile,theresultingsecond-levelsignaturefilesizewouldbeabout5.7MB.

REMOTE DIFFERENTIAL COMPRESSION ALGORITHM OVERVIEW

26

Page 27: Metalogix Replicator for SharePoint - WordPress.com · 4 supports SharePoint Foundation 2010, SharePoint Server 2010, OfficeSharePoint Server 2007 and Windows SharePoint Services

m e t a l o g i x . c o m W H I T E PA P E R

When developing an application that uses RDC, it is important to understand the following concepts and terminology.

InatypicalRDCscenario,aserverandaclienthavedifferentversionsofafile.(Thetermsclientandserverreferonlytothecomputers'rolesinthisscenario,nottheiroperatingsystems.)Theclient'scopyofthefileiscalledtheseedfile.Theserver'scopyiscalledthesourcefile.TheobjectiveoftheRDCapplicationistodownloadthefileupdatestotheclient,whichusesthemtoconstructatargetfilethatcombinestheupdatesfromthesourcefilewiththeunchangedcontentsfromtheseedfile.

TheRDCclientandservereachusetheRDClibrary'sFilterMaxsignaturegeneratortodividetheircopyofthefileintochunksandcomputeastronghash,calledasignature,foreachchunkoffiledata.Thus,theclienthasalistofsignaturesfortheseedfile,andtheserverhasalistofsignaturesforthesourcefile.Thesesignaturelistscanbecomputedonthefly,ortheycanbepre-computed.

Theclient initiatestheRDCprotocolbyrequestingthesourcesignature list fromtheserver.Thentheclientcompares each source signature against the signatures in its own seed signature list. If a source signature matchesaseedsignature,theclientalreadyhasthefiledataforthatsignature.Ifasourcesignaturedoesnotappearintheclient'slistofseedsignatures,theclientmustrequestthespecifiedchunk(offiledata)fromtheserver.

Theresultofcomparingthetwosignaturelistsisaneedslist,whichdescribeswhichchunksoffiledata,fromwhere(seedorsourcefile),areneededtoconstructthetargetfileontheclientcomputer.Eachentry intheneedslistiscalledaneedsblock.

Theclientiteratesthrougheachneedsblockandcopiesthespecifiedchunkofthesourceorseedfiledatatothetargetfile.Seedfiledataiscopiedlocally.Sourcefiledataisdownloadedfromtheserver.Themoresimilartheseedandsourcefilesare,thelessnetworkbandwidthisrequiredtocreatethetargetfile.

REMOTE DIFFERENTIAL COMPRESSION APPLICATION CONCEPTS

Metalogix is the trusted provider of innovative content lifecycle management solutions for Microsoft SharePoint, ExchangeandCloudplatforms.Wedeliverhigh-performancesolutionstoscaleandcost-effectivelymanage,move, store, archive and protect enterprise content. Metalogix provides global support to thousands ofcustomers and strategic partners and is a Microsoft Gold Partner, a managed partner in Microsoft’s High PotentialISVGroupandGSAprovider.MetalogixisaprivatelyheldcompanybackedbyInsightVenturePartnersand Bessemer Venture Partners.

ABOUT METALOGIX

METALOGIX5335WisconsinAveNW,Suite510,WashingtonDC20015

[email protected]|www.metalogix.com|1202.609.9100

27