LAB EVALUATION REPORT DataKeeper Replication for Windows ...it-trendwatch.nl/en/pdfs/DataKeeper Lab Intel SSD report English... · A separate Windows 2012 R2 server was part of the

IT-TrendWatch Evaluation Rapport

1

Copyright © 2015, IT-TrendWatch, All Rights Reserved

LAB EVALUATION REPORT

DataKeeper Replication for Windows Cluster

With Intel Solid State Disk Storage

By Bram Dons

IT Consultant

IT-TrendWatch

January, 2015

www.ftsystems.nl


2


Content

Management Overview ..................................................................................................... 3

DataKeeper Replication for Windows Clusters ............................................................. 4 The SANless Cluster ....................................................................................................... 4

Working DataKeeper Cluster Edition ............................................................................. 4

Replication Types ........................................................................................................... 5

Test configuration DataKeeper ....................................................................................... 6

Installation Windows 2012 R2 Cluster ........................................................................... 8 File Share Witness .......................................................................................................... 8

DataKeeper dummy resource SQL tempdb .................................................................... 9

Installing SQL Server 2012 .......................................................................................... 10

IOmeter Performance Test ............................................................................................ 11 IOmeter Test Results..................................................................................................... 11

Microsoft SQLIO Performance Test ............................................................................. 12 Test results SQLIO ....................................................................................................... 13

TPC–C Database Test ........................................................................................................ 14

HammerDB Load Test Tool ......................................................................................... 14

Comparison Fusion ioDrive2 and Intel SSD DC P3700 ............................................... 15

Conclusion ....................................................................................................................... 16

Appendix A ...................................................................................................................... 17 Cluster Configuration.................................................................................................... 17

Appendix B ...................................................................................................................... 18 Technical Description Intel SSD DC P3700 ................................................................. 18

Appendix C ...................................................................................................................... 20 SQLIO Test results ....................................................................................................... 20

Appendix D ...................................................................................................................... 22 IOmeter Test results ...................................................................................................... 22

The Author ...................................................................................................................... 25

Profile IT-TrendWatch .................................................................................................. 26


3


Management Overview

The company FT Systems has requested IT-TrendWatch to do an investigation into the

DataKeeper replication software from the company SIOS. Data Keeper is a so-called

host-based replication software that provides a Disaster Recovery (DR) solution for the

Linux and Windows environment.

The aim of the study was to see what improvement a performance Solid State Disk

(SSD), especially with the Intel P3700 DC, offers compared with a hard disk drive. The

SQL Server 2012 cluster was used as an application environment for testing. The SQL

Server database transactions where measured with a TPC-C-benchmark tool. During the

tests replication took place of the database files between a primary and secondary server.

Also the input and output transactions (IOPs) and data throughput (MBs) where measured

on the primary server with the P3700 Intel SSD. These values where compared with the

measured values of the Intel 2.5 inch S3700 SSD and SAS Seagate drive.


4


DataKeeper Replication for Windows Clusters

Windows Cluster does not guarantee that application data can be loosed. Windows

Failover Clustering (WSFC) protects the Windows environment and applications only

against the failure of a server node. Although it provides a High Availability (HA)

solution, it does not guarantee against the loss of data in business-critical Windows

applications, such as the SQL Server. This is because in a ' shared-nothing ' cluster

configuration the shared storage is a single point of failure (SPOF). When the shared

storage fails, then none of the cluster nodes have access to the application data. The only

protected files in a Windows cluster are the binary files of the SQL engine that is local to

each server node.

By making one or more copies of the SQL databases the risk of critical loss of database

data can be reduced significantly. This provides a so called Disaster Recovery (DR)

solution. Microsoft provides this through the concept of 'SQL Server Always On’. The

company SIOS provides this with the product Data Keeper Cluster Edition (DKCE), an

alternative for such a replication solution. DKCE is a highly optimized host based

replication solution that seamlessly connects to Microsoft Windows Server 2008 Failover

Clustering (WSFC) and Microsoft Cluster Server (MSCS).

The SANless Cluster

For the SQL Server Always On in combination with WSFC shared storage is required,

for WSFC in combination with DKCE it is not. The arrival of SQL Server 2012 brought

the possibility of using Server Message Block (SMB) 3.0 for creating shares (actually

network based storage). Because of that, there is for shared storage no longer an

expensive, iSCSI or Fibre Channel -based SANs necessary required.

Working DataKeeper Cluster Edition

One of DataKeeper's key components is the kernel mode driver that is responsible for all

mirroring activities between the source and target mirror endpoints. The synchronous or

asynchronous replication takes place on the volume block level. When creating a mirror

first all the data is replicated from the source to the target volume. After the initial

replication (also called “full resync “called ) both volumes are exactly the same. After

that DataKeeper intercepts all writes to the source volume and replicates the data blocks

through the network to the target volume.

Data Keeper uses an intent log file (also referred to as a bitmap file ‘) to store the changes

which are made to the source volume. The log file gives DataKeeper the ability to

recover a failure from the source volume without executing a full mirror resync after the


5


recovery of the failed source server. However, maintaining the log file goes at the

expense of performance that is why it is recommended to place this file on a SSD.

Volumes that cannot be replicated are: Windows system volume, volume (s) with

Windows page file, non-NTFS volumes, non-fixed drive types (CD-ROMs, etc.) and

target volumes that are smaller than the source volume. There is no limit as regards to the

size of the replicated volume.

The built-in WAN optimization makes optimal use of the existing network without using

a WAN accelerator. Through the efficient use of compression algorithms maximum of

the available network bandwidth is used. Data Gatekeeper is able to operate in a 1 GbE

network at a rate of 520 Mbps.

Replication Types Replication of DataKeeper can be applied in various ways. Firstly, by making a second

physical copy of the data. Second, to expand a traditional MSCS/WSFC cluster with a

remote disaster recovery side and take away the spof from the storage of the traditional

MSCS/WSFC cluster.

Data Keeper can be configured in a simple disk replication function of disk-to-disk; one-

to-one, one-to-many and many-to-one. Disk -to-disk replicates data between two volumes

on the same server, one-to-one, one-to-many and many-to-one between one or more

volumes from server (s). The MSCS cluster or WSFC can be expanded with replication

from a shared volume to a remote system via N-shared-disk to one; N-shared-disk to N-

shared-disk or N-shared-disk to multiple N-shared disks.


6


Test configuration DataKeeper

We based our test on a standard two-node Windows Server 2012 R2 cluster. The cluster

was running in an Active Directory Domain that should be installed on a separate DC

server. Both cluster nodes where connected via two networks, a 100Mb LAN and a

10GbE for replication.

On the primary, secondary and backup Windows Servers the DKCE software was

installed. For replication between the two cluster nodes and the Remote Backup Server

Solid State Disks where used. On the cluster SQL Server failover cluster software will be

installed.

As a storage system for the Windows Server operating system on the primary server, the

new Intel PCIe bus based Solid State Disk (SSD) DC P3700 was used with 400GB

storage capacity. On the secondary (backup) server, we used the Intel 2.5 inch SSD DC

S3700 with 400GB.

A separate Windows 2012 R2 server was part of the same domain. On the primary and

secondary Windows Servers DKCE the software was installed. For replication between

the two cluster nodes and the Remote Backup Server SSDs were used. On the cluster was

Microsoft SQL Server 2012 failover cluster software installed.

To test performance of the database, we made use of the HammerDB TPC-C benchmark

test. For separately testing the SSDs we used Intel's IOmeter and Microsoft's SQLIO.


7


Figure 1: Test configuration DataKeeper Cluster Edition based on Intel SSDs


8


Installation Windows 2012 R2 Cluster

For the cluster installation, we follow the standard installation procedure. Because we

configure a SANless cluster and make no use of shared storage, we do not tick 'add all

eligible storage to the cluster’ in the installation menu. Because we do not use a ‘Node

and Majority’ quorum (this is the default choice for a cluster installation), we use instead

the SMB 3.0 from Node and select ‘File Share Majority quorum’.

File Share Witness A File Share Witness must be on a server, in our case the ‘Domain Controller’, being

configured which should not be part of the cluster. The first step is to create a file share

and give our 'mycluster' cluster read/write access on both share and NTFS level. We give

the share the name ‘FSW’.

Now that we have created a file share on the DC, we return to the Primary node and we

use 'Failover Cluster Manager' to change the quorum type. We select 'Configure a file

share witness ' and select the created ‘\\ w2kr2green\fsw’ share. We now have a base 2-

node cluster and proceed on to the next step: the creation of the cluster resources. In

addition, we use a replicated disk resource based on the third-party software ‘Data

Keeper Cluster Edition', this instead of a shared disk resource.

Figure 2: Installing Failover Cluster based on a File Share Witness


9


Installation DataKeeper Cluster Edition

We now have a basic two-node ‘SAN-less' cluster created and we can start installing the

DKCE software on both cluster nodes. A mirror is created between two Solid State

Devices. On the primary host with an Intel SSD DC P3700 on the PCIe bus and an Intel

SSD DC S3700 which is connected to the server SATA controller. To support the NVMe

functionality (see Appendix B) a special by Intel SSD developed driver was used on the

primary node. The SSD drive on the primary node was divided into four partitions:

128GB for the operating system; 128GB for the replication of the SQL database ('r');

50MB for the tempdb SQL files ('t) and 1GB for the bitmap ("x") and on the secondary

node to the SQL database (‘r’), tempdb and bitmap partitions.

DataKeeper dummy resource SQL tempdb

In the new DK-version the possibility exist to add a local volume. For the cluster it looks

like a DK volume, however, with one exception: it is not mirrored. This new DK dummy

resource is very useful in the construction of a SQL cluster because it can be indicated

that tempdb is stored there. When the application is switched (failover) then tempdb

normal was again created by the standby server. The dummy resource solution

accelerated the failover process because it prevents tempdb to be replicated.

First we install DataKeeper on the primary node. In the successive menus, the Domain or

Server account are specified and the license file. After a reboot of the server the

DataKeeper User Interface is started. After typing the primary and secondary server name

a connection to both are made. After clicking the 'Create Job' we follow the menus to

create a mirror on the 'r' drive. In the ‘New Mirror' menu we type the name of the source

volume, the IP address of the server and the 'r' volume. The same input applies to the

Target Server. In the ‘Details’ menu we specify the settings of the mirror, including:

asynchronous or synchronous, the amount of data compressed and the maximum

bandwidth (typically used for WAN replication). We choose for ‘asynchronous’. After

the creation the mirror it can be registered as a volume to WSFC.


10


Figure 3: Mirror details

Installing SQL Server 2012

We continue with the installation of SQL Server 2012 on the primary cluster node. In the

SQL Server Installation Center menu we select "New SQL failover cluster installation. In

the "Instance Configuration menu we give a name to the SQL Server Network. As the

Cluster Disk Selection "tab, we select as shared disk registered Date Keeper Volume 'r', 't'

for the tempdb and 'x' for the bitmaps. In the Network Configuration tab, we specify an IP

address of the primary server. After clicking the "Add node to a Server failover cluster"

we start in the same way the installation on the secondary cluster node.


11


IOmeter Performance Test

To get a first impression of the Intel P3700 DC speed we test it with IOmeter and see to

what extent our results agree with those of the Intel specification. We use the same the

same settings as described in the Intel specification 330566-0005US of October 2014. In

Appendix C the test results are shown.

IOmeter Test Results

When we compare the test results with those of the Intel specification, we measured with

a 4KB random Read/Write a 279.356 IOPs; the Intel specification indicates 450,000

IOPS. That same performance difference also apply to the number of MBs per second:

we tested the Intel SSD with a sequential Read of 2,198 MBs per second, Intel has

measured 2,700 MBs. If we compare the number of IOPs and MBs of the P3700 with that

of a SATA disk then we see huge differences between an SSD and SATA disk. SSD

speeds contrasts with a SATA disk, which shows only 177 IOPs and 55 MBs! More than

2 GBs per second throughput on the P3700 and several hundred thousand IOPs are

unprecedented speeds which were not possible with the previous versions of the PCIe

buses; the difference between the latest PCIe Gen 3x4 and previous versions can be found

in Appendix B.

Figure 4: IOmeter: Random 4KB Read, Queue Depth 32, 4 workers


12


Microsoft SQLIO Performance Test

SQLIO was developed by Microsoft and is a tool that can be used to determine the I/O

capacity of a given storage configuration. The name suggests that it is a SQL Server test,

but that is not the case. It is a test that is similar to Intel's IOmeter.

Comparison Intel SSD DC P3700, S3700 and Seagate SAS Drive

We then tested with Intel's SQLIO benchmark the performance of the Intel SSD DC

P3700, DC S3700 and SAS Seagate Barracuda disk. We performed four tests: write and

read with random and sequential operation. The block lengths varied from 8, 64, and 128

to 256KB on. With the command file FSUTIL createnew r:\testfile.dat (1GB), we first

created a 1GB test file on the fileshare 'r'.

The syntax of the SQLIO command looks like this:

> sqlio –kW –s10 –frandom –o8 –b8 –t2 –LS –Fparam.txt.

> sqlio –kR –s10 –frandom –o8 –b8 –t2 –LS –Fparam.txt.

> sqlio –kW –s10 –sequential –o8 –b8 –t2 –LS –Fparam.txt.

> sqlio –kR –s10 –sequential –o8 –b8 –t2 –LS –Fparam.txt.

In a 'param.txt' file the number of threads and volume testing ('r') was specified. Because

we want to simulate an as good as possible good tuned SQL OLTP environment, we use

two parameters to maintain the disk latency below 5ms in combination with a maximum

performance possible. For this purpose we use the number of threads '-t 'and the number

of outstanding I/O requests ‘-o'.

SQLIO can be used to get the same type of information from an SMB file share. The

simplest way to do is to map a file share to a drive letter using the "NET USE" command

or the PowerShell cmdlet 'New SmbMapping. One can also directly use a UNC path in

the SQLIO command, in place of a drive letter.

For example: SQLIO -s10 -KR -fsequential -b8 -t2 LL -BN \\primary\r $ \testfile.dat.


13


Test results SQLIO

The result of the first SQLIO test is shown in the chart below. It can be seen that the

P3700 performs more than 113,000 IOPs. The S3770 performs with 15,000 IOPs almost

ten times less and the hard disk nearly a thousand times. We see that by increasing the

block length the differences between the sequential test values from the SSD in relation

to the hard disk are getting smaller. This is logical because with a sequential read or write

the mechanical properties of the hard disk play less a role. The other graphs (see

Appendix C) show similar ratios.

Figure 5: SQLIO test


14


TPC–C Database Test

HammerDB Load Test Tool HammerDB is an open source database load testing and benchmarking tool for Oracle,

SQL Server, and other SQL databases. It is an implementation of the TPC-C benchmark

specification but, however, no complete specification. As such, the results cannot be

compared with the officially published TPC-C benchmarks.

HammerDB Benchmark Test

For the TPC-C test, we use HammerDB 2:16 benchmark. We create 128 Warehouses

with 128 virtual users. We take the recommendations of HammerDB 'OLTP Best

Practice. To this end, we check the Server Properties menu the Processor Affinity on

"automatic" is set and the Power Options to "Balanced". After running the test, we find

81,510 Transactions Per Minute (TPM). The network over which the replication is taking

place, shows a load of approximately 11%. The CPU load is less than 17%. The disk

transfer rate is about 9 MBs. Then we disable the replication and restart the benchmark

test and we see approximately the same TPM values. From this we can conclude that the

replication have not or hardly any influence on the performance of the SQL database.

Figure 6: HammerDB test, 128 users


15


Figure 7: Computing Resources HammerDB test

Comparison Fusion ioDrive2 and Intel SSD DC P3700

The company SIOS has a similar PCIe-based SSD with HammerDB test done on the

Fusion ioDrive2; see white paper ‘Enhance Performance and Availability for SQL Server

Deployments with Dell, SanDisk, and SIOS’. The HammerDB test with 128 users Fusion

SSD yielded a mean of 33.048 TPs: our test with the Intel P3700 81.850 TPs.

Microsoft has also tested the ioDrive2, but than in a Hyper-V environment, see: ‘Fusion

ioMemory and Hyper-V workload using Windows Server 2012 Hyper V and SQL Server

2012 Performance Whitepaper’. This paper reported that the HammerDB test had a peak

performance of 50,000 TPs. The conclusion is that the Intel P3700 not only is 1.5 to 2.5

times faster but additionally it comes with a lower price: almost three times per GB less

than the Fusion card. The list price of the Fusion card is 6.599 USD for 785GB the Intel

P3700 cost $ 1.285 for the 450GB card: this yields respectively tot a price of $ 8.4 and $

3.02 per GB storage space. Also the ioDrive2 works on the ‘old’ PCIe Gen2 x 4 bus,

compared to the Intel P3700 which is based on the PCIe Gen3 x4.


16


Conclusion

The Product Data Keeper Cluster Edition of the company SIOS provides with Windows

Cluster a full HA/DR solution for the Windows cluster environment. DataKeeper can be

used in different environments: based on a traditional SAN-based shared storage or via

the new SMB 3.0 Node and File Share Majority quorum of Windows Server 2012 R2.

The latter makes a SANless cluster possible, the environment in which our test was

based.

In general, the application of a Solid State Disk storage system for a SQL Server system

offers an increase in performance. But the use of latest Intel SSD DC P3700 as storage

system, coupled with a PCIe Gen3 bus, gives the SQL Server database and the

DataKeeper replication facility an unprecedented speed. This was demonstrated running a

HammerDB database with TPC-C, SQLIO and IOmeter test on the Intel SSD dc P3700.

In order to demonstrate the stability of the SSD, we performed a failover to either side of

the cluster. This went smoothly. After recovery of the switched off cluster node mirror

was quickly restored and entered no loss of data in the SQL database TPC-C test.


17


Appendix A

Cluster Configuration

PrimaryCluster Node

Dell PowerEdge R420

Internal RAM 64GB DDR3 Processor: 2x Xeon E5 2400 2.27GHz Quad Core

Drives:

C: 400GB Intel SSD DC P3700

E: 400GB SAS Seagate Barracuda Networks:

4x 1GbE: Broadcom 5720

Software:

Windows 2012 R2 Datacenter DataKeeper version 8.2.1

SQL Server 2012 Enterprise Edition

Secondary Cluster Node Motherboard: Intel Server S5500HCV

Internal RAM 32GB DDR3

Processor: Xeon E5520 2.27GHz Quad Core

Drives: C: 400GB SSD drive Intel SSD DC S3700 Series

Netwerken:

1GbE: Intel 82575EB 10GbE: 82598EB

Software:

Windows 2012 R2 Datacenter

DataKeeper version 8.2.1 SQL Server 2012 Enterprise Edition

Domain Controller

Motherboard: Intel Server S3200SH Internal RAM 8GB DDR3

Processor: Xeon 3210SHLX 2.27GHz Quad Core

Drives:

C: 500GB RAID 0, SATA Hitachi Deskstar Netwerken:

1GbE: Intel 82575EB

10GbE: 82598EB

Software:

Windows 2012 R2 Datacenter

SQL Server 2012 Enterprise

Benchmark Factory Database Intel IOmeter

HammerDB


18


Appendix B

Technical Description Intel SSD DC P3700

The Solid-State Drive Data Center for PCIe Family consists of the SSD DC 3500, 3600

and 3700 Series. They are on NAND flash memory multi-level cell (MLC) -based SSDs.

As an interface the PCIe 3.0 x 4 bus can be used. The Intel SSD Data Center Family for

PCIe provides extremely high data throughput to Intel Xeon processors, up to six times

higher data transfer speeds than 6 Gbps SATA/SAS-based SSDs. The DC Family Intel

SSDs are able to read up to 2.8 GBs and 460K IOPs write 2.0 GBs and 175K IOPs. The

P3700 Series performance can match multiple 15K Hard drives with each 200 IOPs. But

for that alone 2300 hard disk drives are required!

The DC series are available in half-height, half-length slot configuration.

Using Intel-enabled software drivers NVMe supports the major operating systems,

including: Microsoft Windows 2008/2012, Red Hat 6.5, UEFI 2.3.1 and Linux. The

P3700 is available in 400GB, 800GB, 1.6TB and 2TB version. The power consumption is

respectively 12, 18, 22 and 25 watts. The expected life is 2 million hours MTBF, 230

years. The Read/Write latency is 20/20µs, the power consumption is 20W.

Intel SSD DC P3700


19


Non-Volatile Memory Express (NVMe)

The NVMe protocol is much faster and has a much lower latency than the SATA or SAS

protocols. An on NVMe based storage device is directly connected to the PCIe lanes and

is physically closer to the CPU. This results in a very low latency. SATA is designed for

mechanical hard drivers, NVMe is intended for flash. Perhaps the most valuable

contribution for SSD support is the PCIe Gen 3x4 interface. Previously SSDs used the

PCIe Gen 2 interface, which only provides a theoretical maximum of 500MBs per Lane,

or 2GBs at an x4 device (4 lanes). The new PCIe Gen 3 delivers 8 GBs per Lane, with a

total of nearly 4TBs!

When reading reviews, IOPs are usually measured under various queue depths. Normally

that is 32 outstanding commands. That is the point at which most SATA-based SSDs

deliver their maximum peak performance; it is also limited by AHCI. Many flash

controllers support larger queue depths. This can be seen when a PCIe-based SSD uses

the proprietary drivers which are supplied by the supplier; for this reason Intel developed

for the SSD DC P3700 their own drivers. NVMe with not only increases the commands

per queue from 32 to 64,000 but also the number of queues from 1 to 64,000.


20


Appendix C

SQLIO Test results


21



22


Appendix D

IOmeter Test results

Prestatietest Intel Solid State Drive P Series met Iometer

Type Drive Intel SSD DC P3700 Intel Product Specificatie Seagate SATA

DC P3700 October 2014, 330566-005US Barrucuda 7200

Random Read/write IOPs

Random 4KB 70/30 Read/Write 116,119 150,000 177

Random 4KB Read 279,356 450,000 185

Sequential Read en Write Bandwith MBs

Sequential Read 128KB 2,198 2,700 55

Sequential Write 128KB 1053 1,080 56

IOmeter instellingen

Random 70/30 Read/Write : 4 Workers, Queue Depth 32 en 4 KB transfer size.

Sequential Read: 1 Worker, Queue Depth 128 en 128KB transfer size.

Sequential Write: 1 Worker, Queue Depth 128 en 128KB transfer size.

IOmeter settings according the Intel Specification 330566-005US


23


Random Read/Write 70/30, 4KB Block Length, 32 queue Depth, 4 Workers

Random Read, 4 KB Block Length, 32 Queue Depth, 4 Workers


24


Sequential Read, 128KB Block Length, 128 Queue Depth, 1 Worker

Sequential Write, 128KB Block Length, 128 Queue Length, 1 Worker


25


The Author

Bram Dons has been working as an independent IT analyst at IT-TrendWatch.

He studied Telecommunications and Technical Management at the Technical College.

For several decades, he held various positions in the IT industry, including technical

management, software and hardware engineering. In 2000 he founded the IT research firm IT-TrendWatch, of which he is director. The

company conducts research and provides advice in the field of new IT technology and IT

architecture in both hardware and software. In addition to research, the company

conducts evaluations in the field of storage, system high availability, virtualization and

cloud computing. He is author of several IT books and writes for many leading journals

and textbooks in the IT field (see www.it-trendwatch.nl).


26


Profile IT-TrendWatch

IT-TrendWatch is an independent technology research company.

The company is specialized in the evaluation of software and hardware products related

to the distributed IT environment, cluster, virtualization and storage market.

IT-Trend Watch has its own test laboratory which that consist of a SAN, NAS and ISCSI

storage systems. Test platforms are Windows NT, Linux, Hyper-V, VMware and Citrix.

The company has an extensive background with over thirty years’ experience in the

computer industry in the software design, hardware engineering and technical

management.

Companies can for future investments in new IT technology seek advice and brought into

contact with relevant companies. IT-TrendWatch also performs benchmark tests for new storage and virtualization

platforms and cluster environments.

The company publishes articles and review reports and makes blogs for various Dutch IT

magazines and web sites. On request they give seminars and presentations on new IT

technology.

For more information email us at [email protected] or call us +31 (0) 36 5328569

Documents

LAB EVALUATION REPORT DataKeeper Replication for Windows ...it-trendwatch.nl/en/pdfs/DataKeeper Lab Intel SSD report English... · A separate Windows 2012 R2 server was part of the