Deploying the Elastic Stack on Dell EMC PowerFlex family

000060

White Paper

Deploying the Elastic Stack on Dell EMC PowerFlex family

Abstract This white paper highlights the deployment of Elastic Stack on the Dell EMC PowerFlex family and provides few best practices.

June 2020

Revisions

2 Deploying the Elastic Stack on Dell EMC PowerFlex family | 000060

Revisions Date Description

March 2020 Initial release

June 2020 Updated VxFlex to PowerFlex as per new re-branding guidelines

Acknowledgements Author: Kailas Goliwadekar

The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any software described in this publication requires an applicable software license. Copyright © 2020 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be trademarks of their respective owners. [6/19/2020] [White Paper] [000060]

Table of contents


Table of contents Revisions............................................................................................................................................................................. 2

Acknowledgements ............................................................................................................................................................. 2

Table of contents ................................................................................................................................................................ 3

Executive summary ............................................................................................................................................................. 5

1 Introduction ................................................................................................................................................................... 6

1.1 Objective ............................................................................................................................................................. 6

1.2 Audience ............................................................................................................................................................. 6

1.3 Terminology ........................................................................................................................................................ 7

2 Product overview .......................................................................................................................................................... 8

2.1 PowerFlex family ................................................................................................................................................ 8

2.2 PowerFlex software components ....................................................................................................................... 8

2.2.1 PowerFlex .................................................................................................................................................. 8

2.2.2 PowerFlex Manager ................................................................................................................................... 9

2.3 PowerFlex consumption options ......................................................................................................................... 9

2.3.1 PowerFlex rack ........................................................................................................................................... 9

2.3.2 PowerFlex appliance .................................................................................................................................. 9

2.3.3 VxFlex Ready Nodes .................................................................................................................................. 9

2.4 PowerFlex deployment architectures ................................................................................................................. 9

2.4.1 Two-layer (Server SAN) architecture ....................................................................................................... 10

2.4.2 Single-layer (HCI) architecture ................................................................................................................. 10

2.4.3 Storage-only architecture ......................................................................................................................... 10

2.5 Elastic Stack ..................................................................................................................................................... 10

3 Elastic Stack architecture overview ............................................................................................................................ 12

3.1 Elasticsearch .................................................................................................................................................... 12

3.2 Logstash ........................................................................................................................................................... 13

3.3 Kibana ............................................................................................................................................................... 14

3.4 Beats ................................................................................................................................................................. 14

4 Solution architecture ................................................................................................................................................... 15

4.1 Logical architecture .......................................................................................................................................... 15

4.2 Network architecture ......................................................................................................................................... 16

5 Testing and Validation ................................................................................................................................................ 17

5.1 Test methodology ............................................................................................................................................. 17

5.2 Test results ....................................................................................................................................................... 18

Table of contents


6 Best practices ............................................................................................................................................................. 20

6.1 PowerFlex rack ................................................................................................................................................. 20

6.2 PowerFlex ......................................................................................................................................................... 20

6.3 PowerFlex network ........................................................................................................................................... 20

6.4 Elasticsearch .................................................................................................................................................... 20

7 Deployment of the Elastic Stack on PowerFlex ......................................................................................................... 21

8 Conclusion .................................................................................................................................................................. 22

A Appendix A: Detailed results ...................................................................................................................................... 23

A.1 Use case1: EventData track ............................................................................................................................. 23

A.2 Use case1: Http_logs track ............................................................................................................................... 25

B Appendix B: Configuration details .............................................................................................................................. 28

C Technical support and resources ............................................................................................................................... 29

C.1 Related resources ............................................................................................................................................ 29

C.2 Additional resources ......................................................................................................................................... 29

Executive summary


Executive summary Teams responsible for managing large-scale and rapidly growing data center infrastructure constantly meet challenges in terms of scalability and flexibility needs. Enterprises are transitioning from the traditional data center to a hyperconverged infrastructure (HCI) approach to address these challenges. HCI based offerings give enterprises the ability to support modern workload and provides a simplified solution to manage their infrastructure needs. Organizations are switching to public cloud security and log analytics to provide uninterrupted operations with minimum downtime and SLAs with huge amount of data being generated daily. Since many organizations are switching to cloud, the necessity for public cloud security tools and log analytics platforms is increasing rapidly.

A substantial amount of this data consists of logs like web server, http, application logs, and so on. With careful and detail log analysis, organizations can tap on various business opportunities and threats surrounding it. The Elastic Stack solution on Dell EMC PowerFlex family provides complete log analysis that helps in searching, analyzing, visualizing data from different machines, in addition helping to identify any potential security threats.

This white paper provides guidelines and best practices for optimal deployment of Elastic Stack on Dell EMC PowerFlex family.

The Dell EMC PowerFlex family for the Elastic Stack offers unique advantages. The results from Elastic Stack performance benchmarking tests, executed using benchmarking tool Rally clearly showed that PowerFlex can support the performance requirements of the Elastic Stack. Other key benefits of deploying the Elastic stack on PowerFlex include:

• Flexibility of deployment options. • Faster indexing and searching performance. • Easily scalable solution to multiple Elastic master and data nodes

Introduction


1 Introduction This white paper outlines the design consideration of deploying the Elastic Stack on PowerFlex family. The primary focus of this paper is to provide the reader with details of an Elastic Stack deployment on PowerFlex family. This paper also provides more details on the best practices while deploying the Elastic Stack on PowerFlex along with some performance benchmarking results using Rally tool.

1.1 Objective This white paper demonstrates

• PowerFlex family and Elastic Stack Overview • Elastic Stack Architecture • Deployment of Elastic Stack on PowerFlex rack • Validating Elastic Stack on PowerFlex rack • Conclusion

1.2 Audience This white paper is intended for Elastic Stack administrators, system engineers, partners, and members of Dell EMC and partner professional service community who are looking to make real-time decisions, and analysis of data to make the right marketing decisions.

The reader of this document must have a working knowledge of the following technologies:

• Dell EMC PowerFlex portfolios • Elastic Stack • VMware vSphere

Introduction


1.3 Terminology The following table defines acronyms and terms that are used throughout this document:

Terminology

Term Definition

HCI Hyperconverged Infrastructure

OOB Out-of-band management

SVM Storage Virtual Machine

SDS Storage Data Server

SDC Storage Data Client

VLAN Virtual Local Area Network

TOR Top of the Rack Switch

Product overview


2 Product overview

2.1 PowerFlex family PowerFlex is a software-defined storage platform designed to significantly reduce operational and infrastructure complexity, empowering organizations to move faster by delivering flexibility, elasticity, and simplicity with predictable performance and resiliency at scale. The PowerFlex family provides a foundation that combines compute as well as high performance storage resources in a managed unified fabric. PowerFlex comes in flexible deployment options - rack, appliance or ready nodes - that enable two-layer (compute & server SAN), single-layer (HCI), and/or storage only architectures. PowerFlex is ideal for high performance applications and databases, building an agile private cloud, or consolidating resources in heterogeneous environments.

PowerFlex family

2.2 PowerFlex software components Software is the key differentiation and the “secret sauce” in the PowerFlex offering. PowerFlex software components not only provide software-defined storage services, they also help simplify infrastructure management and orchestration with comprehensive ITOM and LCM capabilities that span compute as well as storage infrastructure, from BIOS and Firmware to nodes, software and networking.

The core foundational component in the PowerFlex family that enables Software Defined Storage (SDS) services is called simply PowerFlex, to represent the core value it enables for the platform. Additionally, PowerFlex Manager is a comprehensive IT Operational Management (ITOM) and Life Cycle Management (LCM) tool that drastically simplifies management and ongoing operation.

2.2.1 PowerFlex PowerFlex (previously VxFlex OS) is the software foundation of PowerFlex software-defined storage. It is a scale-out block storage service designed to deliver flexibility, elasticity, and simplicity with predictable high performance and resiliency at scale.

Product overview


PowerFlex management is available using a GUI, CLI, and REST clients. There is a VMware vSphere® plug-in that allows VMware admins to deploy, upgrade, configure, and manage PowerFlex in an ESXi environment within VMware vSphere.

2.2.2 PowerFlex Manager PowerFlex Manager is the software component in PowerFlex family that enables ITOM automation and lifecycle management capabilities for PowerFlex systems.

2.3 PowerFlex consumption options PowerFlex SDS platform is available in multiple consumption options to help customers meet their project and datacenter requirements. PowerFlex appliance and PowerFlex rack provide customers comprehensive IT Operations Management (ITOM) and lifecycle management (LCM) of the entire infrastructure stack in addition to sophisticated high-performance, scalable, resilient storage services. PowerFlex appliance and PowerFlex rack are the two preferred and proactively marketed consumption options. PowerFlex is also available on VxFlex Ready Nodes without the ITOM and LCM capabilities.

Note: The brand for Ready Nodes continues to be VxFlex.

2.3.1 PowerFlex rack PowerFlex rack is a software-defined storage platform designed to deliver flexibility, elasticity, and simplicity with predictable performance and resiliency at scale by combining compute as well as high performance storage resources in a managed unified network. This rack-scale engineered system, with integrated networking, enables customers to achieve the scalability and management requirements of a modern data center.

2.3.2 PowerFlex appliance PowerFlex appliance is a software-defined storage platform designed to deliver flexibility, elasticity, and simplicity with predictable performance and resiliency at scale by combining compute as well as high performance storage resources in a managed unified network. This turnkey offer allows customers the flexibility and savings to bring their own compatible networking. With PowerFlex, customers deploy to match their initial needs and easily expand with massive scale potential, without having to compromise on performance and resiliency.

2.3.3 VxFlex Ready Nodes VxFlex Ready Nodes are validated server building blocks configured for use with PowerFlex. They are available with thousands of configuration options and are available for customers who prefer to build their own environments.

2.4 PowerFlex deployment architectures PowerFlex software-define storage offers flexibility of deployment architecture to help best meet the specific deployment and architectural requirements. PowerFlex can be deployed in a two-layer (Server SAN), single-layer (HCI), or in storage-only architectures.

Product overview


2.4.1 Two-layer (Server SAN) architecture In this White Paper solution, PowerFlex is deployed using two-layer architecture.

In two-layer architecture, nodes that provide storage capacity and host datasets are separated from nodes that host applications and workloads. PowerFlex manager provides LCM and IOTM for the entire infrastructure, including nodes that provide storage and nodes that host the applications. Compute and storage resources can be scaled by adding respective node to the cluster. This segregation of compute and storage resources can be helpful to minimize software licensing costs in certain situations. This architecture could be most suitable for hosting high-performance high-value databases and application workloads.

2.4.2 Single-layer (HCI) architecture In this architecture, each node in the cluster contributes storage resources as well as hosts applications and workloads. This architecture allows you to scale your infrastructure uniformly and with a pre-defined building block that adds both storage and compute resources. PowerFlex Manager provides ITOM and LCM capabilities for the entire infrastructure. This architecture is most suitable for datacenter and workload consolidation.

2.4.3 Storage-only architecture In this architecture, PowerFlex provide only storage resources. Using PowerFlex storage-only nodes, a software-defined block storage environment is created that can be accessed and consumed by a number of applications and workloads that are hosted outside of PowerFlex cluster. PowerFlex Manager provides LCM and ITOM for the storage infrastructure. This is a suitable architecture where customer has existing compute infrastructure but needs high-performance SDS. This can be a starting point with the customer, and may expand to a two-layer Server SAN deployment in the future as the external compute is migrated to PowerFlex.

2.5 Elastic Stack The Elastic Stack is a group of open source products from Elastic. It was previously known as the ELK Stack. It is an amazing and powerful collection of three open source projects namely Elasticsearch, Logstash, and Kibana. These three technologies work well with each other despite starting out as separate projects.

Elastic Stack components

Product overview


The Elastic Stack is widely known as the de facto way to do end-to-end log analysis solution and helps in deep searching, analyzing, and visualizing the logs generated from different machines. The Elastic Stack also includes turnkey solutions focused on in three broad level cases:

• Enterprise Search -- application search, site search, and workplace search. • Observability -- logging, APM, and metrics. • Security -- SIEM, endpoint, and security analytics.

Value added features such as machine learning, alerting, maps services, canvas, cross cluster replication, cross cluster search, field and document level security, and cluster authentication can all be leveraged to help most of the information about your data. Many of these capabilities are free and others can be licensed through an Elastic Subscription that includes support. For more information, contact the Elastic team.

https://www.elastic.co/subscriptions

https://www.elastic.co/contact?storm=global-header-en

Elastic Stack architecture overview


3 Elastic Stack architecture overview The Elastic Stack is designed to help users to import data with any format and source; to search, analyze, and visualize imported data in real time. The following figure displays the components available in the Elastic Stack:

• Beats are agents that ship the data from different systems.

• Logstash collects logs, and events data. It parses, filters, and transforms the data.

• Elasticsearch stores, searches, and analyzes the data.

• Kibana is used to visualize the data.

Elastic Stack architecture

3.1 Elasticsearch Elasticsearch is a NoSQL database that is based on Lucene search engine and provides a restful interface for different operations. It is easy to scale, flexible, schema-less distributed search and analytics engine. It provides simple deployment maximum reliability and uses indexes for data storage. Elasticsearch can be easily scaled horizontally. It is used to perform detailed analysis and stores the data centrally for quick search of the documents.

An Elasticsearch cluster is a collection of one or more nodes (servers) that together holds the entire data and provides federated indexing and search capabilities across all nodes. A node is a single server that is part of the cluster, stores the data, and participates in the cluster’s indexing and search capabilities.



An index is a collection of documents that have similar characteristics. A document is a basic unit of information that can be indexed. When an index is created, the number of shards can be defined. Each shard is a fully functional and independent "index" that can be hosted on any node in the cluster. Sharding is important for two primary reasons:

• It allows horizontally split or scale the content volume. • It allows to distribute and parallelize operations across shards (potentially on multiple nodes) thus

increasing performance/throughput.

For more information about Elasticsearch, see Elastic Stack and Product Documentation .

3.2 Logstash Logstash is the data collection pipeline tool through which input is taken from different sources and output to different data sources. The data is cleansed through filter options and transformed before sending it to Elasticsearch. Logstash has different adapters to handle different applications, and it normalizes the data into different destinations. Processing in Logstash is organized into one or more pipelines.

Logstash

These pipelines receive data and place it in internal queue. The processing threads then use filter plugins, and once the data is processed its send to the output plug-in.

https://www.elastic.co/guide/en/elasticsearch/



3.3 Kibana Kibana is an open-source analytics software data visualization tool. With this tool, different visualizations, charts, maps, histograms, and dashboards can be created. Kibana reads data from Elasticsearch and helps developers to have an immediate insight into it. Kibana enables creating and saving custom dashboards. It can be used for search, view, and interact with data that is stored in Elasticsearch directories. There are different methods for performing searches on the data.

A sample Kibana dashboard is shown here:

Sample Kibana Dashboard

The most common search types in Kibana are Free text searches, Field-level searches, Logical statements, and Proximity searches.

Kibana offers loads of advantages like easy visualizing, real-time analysis, debugging capabilities, saving and managing multiple dashboards, and it is completely integrated with Elasticsearch.

3.4 Beats Beats are open-source data shippers that are installed as agents on the servers and send operational data to Elasticsearch. They are lightweight and can send the data from hundreds or thousands of machines to Elastic Common Schema (ECS) that normalizes the data. Each Beat is a separately installable product.

For example, Metricbeat is used to collect metrics for memory usage, CPU usage, and disk space, whereas Filebeat is used to send file data such as logs.

Solution architecture


4 Solution architecture For this paper, the Elastic Stack is deployed on Dell EMC PowerFlex rack. The PowerFlex integrated is configured in a two-layer configuration meaning that servers are either compute nodes, providing the ESXi hypervisor or storage nodes running PowerFlex.

4.1 Logical architecture The below diagram shows logical architecture diagram of the Elastic Stack on Dell EMC PowerFlex rack two-layer setup that had four SDS and three SDC. The setup of PowerFlex two-layer was configured using PowerFlex Manager which automates the entire deployment process.


From PowerFlex standpoint, there was a single PowerFlex cluster with a protection domain, consisting of four RHEL storage only nodes that played the role of SDS. Each host is populated with ten 1.92 TB Toshiba SSD drives. These 40 disks are used to create a storage pool from which the usable storage volumes are created.

The PowerFlex client software (SDC) is installed on each ESXi host in order to access the volumes created in the storage pool, a VMware Datastore was created on each of these volumes. The Elastic Cluster VMs were deployed onto these datastores. The Elastic cluster that is included one Master VM and two Data VMs.

On each of the three SDCs, a single CentOS VM was created. The first to host Logstash, the second Elasticsearch and the third Kibana. All these three VMs could communicate with each other.

For detailed configuration of PowerFlex integrated nodes, see Appendix 9.2 Configuration.



4.2 Network architecture The following figure demonstrates high-level network architecture of the two-layer set up on PowerFlex rack system:

Network architecture

PowerFlex rack networking details Components Description

Cisco Nexus 93180YC-EX 10 Gbps & 25 Gbps TOR switches Cisco Nexus 9332PQ 25 Gbps Aggregation switches Cisco Nexus 3172TQ 1 Gbps & 10 Gbps Management switches PowerFlex storage traffic 2 x 25 Gbps links

Testing and Validation


5 Testing and Validation This section provides a detailed summary and description of the tests that are performed to validate the Elastic Stack hosted on PowerFlex rack with benchmarking tool Rally.

5.1 Test methodology Unit testing through System testing methods were used to perform various tests on the Elastic Stack. The Elastic team used Rally, a benchmarking tool to run their nightly benchmarking tests.

Rally acts as load generator tool for Elasticsearch. It also builds, sets up and tears down the Elasticsearch cluster. This makes Rally a suitable tool to run in a basic environment. Rally also can be used on an existing Elasticsearch cluster to manage benchmark configurations, compare results, and find any performance issues. For more information about Rally, see Getting Started with Rally.

For this paper, most prevalent use cases of Elastic were chosen that demonstrates the PowerFlex family is one of the best platforms to host Elasticsearch.

Use Case 1: This use case is based on the rally-eventdata-track that simulates event-based data use-cases. This track can be used to create more complex and realistic simulations and benchmarks. More information about this track use can be found at out at https://github.com/elastic/rally-eventdata-track.

Use Case 2: This use case is based on the webserver logs that are collected from a big event in 1998. These are real logs and demonstrate Elasticsearch indexes these logs and can be visualized in Kibana. More information about this track can be found out at https://github.com/elastic/rally-tracks/tree/master/http_logs.

https://esrally.readthedocs.io/en/stable/quickstart.html

https://github.com/elastic/rally-eventdata-track

https://github.com/elastic/rally-tracks/tree/master/http_logs



5.2 Test results Rally was run on a separate host to benchmark the Elasticsearch cluster hosted on a two-layer PowerFlex cluster. The two-layer configuration included three compute nodes and four storage nodes. The Elasticsearch cluster includes one Master Node and two Data Nodes. The following figure shows the test set up for Rally:

Test setup

Use Case 1: A challenge ‘lasticlogs-1bn-load from EventData track was chosen to benchmark existing the Elasticsearch cluster. This challenge indexes 1 billion events into several indexes of two primary shards each and results in around 200 GB of indexes being generated on disk. It can be used give an idea of how max indexing performance behaves over an extended period. The benchmarking was done using the following command:

esrally --track=eventdata --target-hosts=192.168.105.71:9200,192.168.105.72:9200,192.168.105.73:9200 --track-repository=eventdata --challenge=elasticlogs-1bn-load --pipeline=benchmark-only



Results for challenge elasticlogs-1bn-load

Use Case 2: http_logs track was chosen and run using Rally. This is rather a simple track based on the webserver logs from the 1998 Football world cup. This track has five shards, and eight bulk indexing requests are issued when the test is running. The benchmarking was done using the following command:

esrally --track=pmc --target-hosts=192.168.105.71:9200,192.168.105.72:9200,192.168.105.73:9200 --pipeline=benchmark-only

The key results for track http_logs are represented in the following table:

Results for track http_logs

For detailed results, see Appendix A.1.

Parameters Value

Median Throughput for index-append-1000-elasticlogs_q_write 93,673 docs/s

Median Throughput for node_storage 10.02 ops/s

Error Rate 0%

Total time to run the tests 10,844 s

Average CPU Utilization on each Data Node 58%

Memory Utilization on each Data Node 32%

Network Utilization on each Data Node 96,000 KBps

Parameters Value

Index Throughput 302,283 docs/s

Cumulative indexing time of primary shards 146 min

Total Young Gen GC 140.62 s

Total time to run the tests 90.31 min

Average CPU Utilization on each Data Node 30%

Memory Utilization on each Data Node 25%

Best practices


6 Best practices The following sections outline the best practices followed in this solution.

6.1 PowerFlex rack • Minimum number of nodes for production workload is seven for a PowerFlex two-layer set up. • Homogenous node types are recommended for predictable performance. • Maximum number of devices in a storage pool is 300. • Maximum number of nodes in a protection domain is 32. • Change the passwords for all default accounts. • Use secure communication – HTTPS (TCP port 443) to remotely access PowerFlex nodes. • Ensure PowerFlex rack is compliant to an RCM.

6.2 PowerFlex • Configure high-performance profile for MDM, SDS, and SDC. • Disable Read Flash cache and Read RAM cache for all flash clusters. • Check with PowerFlex platform team to increase the per device queue length value to 256 per host

for improving the I/O concurrency. • Ensure that the customize power plan is set to High Performance.

6.3 PowerFlex network • Confirm with PowerFlex platform team to enable Jumbo frames for Windows VM, SVM, and at ESXi

host. • Use the Para virtual SCSI (PVSCSI) controller on guest VMs to achieve high performance. • Enable secure network protocol options only (for example, HTTPS and Secure Shell (SSH). • Separate management and control traffic from production application traffic. You can provide this

separation by using VLANs. • Separate VMware vSphere vMotion traffic from production traffic according to PowerFlex standard.

6.4 Elasticsearch • RAM to be 32 GB or 64 GB for a virtual machine. • JVM Heap size to 50% of RAM, max up to 32 GB. Keep at least 50% RAM available. • If Elasticsearch must scale vertically, then add more vCPUs. • Start with a proof of concept, and then test, optimize, iterate, and scale. • Elasticsearch recommends increasing ulimit of File descriptors to 65536. • Disable swap file. • Increase the refresh interval of index from 1 s to more. For example, 20 s. This interval depends

on case to case basis. • Benchmarking VM where Rally is supposed to run on a different VM which is not part of the

Elasticsearch cluster. • Seven Tips for Better Elasticsearch Benchmarks

Note: These best practices were used for the Elasticsearch configuration on a virtual machine and few choices made were subjective.

https://www.elastic.co/blog/seven-tips-for-better-elasticsearch-benchmarks.

Deployment of the Elastic Stack on PowerFlex


7 Deployment of the Elastic Stack on PowerFlex

The Elastic Stack can be deployed on different operating systems. For this white paper, Elastic Stack is deployed on CentOS 7 VM on Dell EMC PowerFlex rack.

Elastic cluster deployment had one Master Node, and two Data Nodes. All these clusters were deployed as Virtual Machines with CentOS 7 on different ESXi hosts.

1. Installed Elasticsearch by following the procedure in the following link https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html

2. Once Elasticsearch is installed, then Logstash was installed using the procedure in the following link https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

3. Kibana was installed to access the dashboards using the procedure in the following link

https://www.elastic.co/guide/en/kibana/current/rpm.html

4. Installed other VMs with Filebeat to push log file into the Elasticsearch. Beats were installed using the procedure in the following link https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html

https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

https://www.elastic.co/guide/en/kibana/current/rpm.html

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html

Conclusion


8 Conclusion The solution provided in this paper, demonstrates how you can deploy the Elastic Stack on PowerFlex family to meet performance, resiliency, and scale. In addition, it states the best practices for deployment of PowerFlex system and the Elastic Stack. The solution showcases how PowerFlex can address some of the key Elasticsearch application challenges, for example the benchmark results demonstrate the capability of indexing one billion documents in approximately 3 hours, highlighting the extreme performance capabilities of PowerFlex.

Elastic customers are looking for extreme performance and scalability and flexibility to be deployed in multiple operating environments such as Windows, Linux, Virtualized or Bare Metal. PowerFlex provides the scalable compute and storage that is required for the Elastic stack with utmost flexibility.

Besides, Elastic is a GCP partner, available through the GCP Marketplace and PowerFlex is already validated as the infrastructure for Google Anthos solution.

Appendix A: Detailed results


A Appendix A: Detailed results

A.1 Use case1: EventData track The challenge elasticlogs-1bn-load was run on EventData track and the results are captured in this section.

PowerFlex Dashboard during test run

CPU Utilization (60%) on one of the Data Nodes, the other data node had similar CPU Utilization.



Memory Utilization (47%) on Data Node

Network Utilization on Data Node

Iostat on Data Node



A.2 Use case1: Http_logs track

The http_logs_track was run with Rally, and the results are captured in this section.

PowerFlex Dashboard during test run

CPU Utilization on one of the Data Nodes, the other data node had similar CPU Utilization



Memory Utilization on Data Node

Network Utilization



Iostat on Data Node

Appendix B: Configuration details


B Appendix B: Configuration details Configuration details

Component Description

PowerFlex rack Compute Nodes

3 x PowerFlex Nodes (R640 servers):

• VxFlex OS* version: R3_0.1 • ESXi version: 6.7 U2 • CPU: 2 x Intel(R) Xeon(R) Gold 6140 CPU @ 2.30 GHz, 18 cores • Memory: 384-GB RAM (12 x 32-GB DIMMs)

4 x PowerFlex Nodes (R640 servers):

• VxFlex OS* version: R3_0.1 • VMware ESXi version: 6.7 U2 • CPU: 2 x Intel(R) Xeon(R) Gold 6126 CPU @ 2.60 GHz, 12 Cores • Memory: 192-GB RAM (12 x 16-GB DIMMs)

*Note: VxFlex OS rebranded as PowerFlex from version 3.5.

Network 2 NIC cards, each having 2 ports 25 GbE connection

Elastic Stack • Elasticsearch: 7.3.2 • Kibana: 7.3.2 • Logstash:7.3.2 • FileBeat: 7.3.2 • Rally: 1.3

Elasticsearch Master VM • Operating system version: CentOS7 • vCPU: 16 • Memory: 32 GB • HeapSize: 16 GB

Elasticsearch Data Node VM • Operating system version: CentOS7 • vCPU: 16 • Memory: 64 GB • HeapSize: 32 GB

Rally VM • Operating system version: CentOS7 • vCPU: 8 • Memory: 16 GB

Technical support and resources


C Technical support and resources

C.1 Related resources See the following referenced or recommended resources that are related to this document:

Note: The following links are open to customers although some may require registration for access.

• PowerFlex blog: https://blog.dellemc.com/en-us/tag/PowerFlex/ • Elasticsearch: https://www.elastic.co/products/elasticsearch • Rally: https://esrally.readthedocs.io • Elasticsearch Tuning: https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-

indexing-speed.html • Deployment of Elastic Stack: https://www.howtoforge.com/tutorial/how-to-install-elastic-stack-on-

centos-7/

C.2 Additional resources • Dell.com/support is focused on meeting customer needs with proven services and support. • Dell Technologies InfoHub: Technical White paper and Workload solutions.

https://blog.dellemc.com/en-us/tag/scaleio/

https://www.elastic.co/products/elasticsearch

https://esrally.readthedocs.io/

https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html

https://www.howtoforge.com/tutorial/how-to-install-elastic-stack-on-centos-7/

https://www.howtoforge.com/tutorial/how-to-install-elastic-stack-on-centos-7/

http://www.dell.com/support

https://infohub.delltechnologies.com/

Documents

Deploying the Elastic Stack on Dell EMC PowerFlex family