Predix Logging

© 2020 General Electric Company

Contents

Pedix Logging Overview 1

About Predix Logging 1

Predix Logging Subscription Plans 1

Getting Started with Predix Logging 3

Creating Predix Logging Service Instance 3

Setting up the Predix Logging Kibana Interface 4

Using Predix Logging 5

Data Persistence and Back for Predix Logging 5

Migrating from an Existing Service or Plan to a New Service or Plan 5

Troubleshooting Predix Logging 5

ii Predix Logging

Copyright GE Digital© 2020 General Electric Company.

GE, the GE Monogram, and Predix are either registered trademarks or trademarks of General ElectricCompany. All other trademarks are the property of their respective owners.

This document may contain Confidential/Proprietary information of General Electric Company and/or itssuppliers or vendors. Distribution or reproduction is prohibited without permission.

THIS DOCUMENT AND ITS CONTENTS ARE PROVIDED "AS IS," WITH NO REPRESENTATION ORWARRANTIES OF ANY KIND, WHETHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TOWARRANTIES OF DESIGN, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. ALL OTHERLIABILITY ARISING FROM RELIANCE UPON ANY INFORMATION CONTAINED HEREIN IS EXPRESSLYDISCLAIMED.

Access to and use of the software described in this document is conditioned on acceptance of the EndUser License Agreement and compliance with its terms.

© 2020 General Electric Company iii

Pedix Logging Overview

About Predix LoggingLearn about the Predix Logging service

Use the Predix Logging service to collate and view your log files. Predix Logging is the cloud-enabledversion of the ELK stack http://www.logsearch.io/. ELK is a tool for receiving, processing and outputtinglogs. ELK uses Elasticsearch as a back-end datastore, and kibana as a front-end reporting tool to create apowerful pipeline for storing, querying and analyzing your logs.

The Predix Logging service is based on several open-source software components:

Elastic SearchIndexes and stores the log data.

ELKReceives and parses the log data.

KibanaProvides a customizable, user-friendly way to visualize your log data.

Predix Logging Subscription PlansLearn about your options for using the Predix Logging service.

Predix Logging subscription plans offer the following:

Dedicated Single NodePreferred for higher memory with non-HA requirements. This could be used for production use casesif the application is designed for resiliency.

Dedicated and Highly Available (HA)Best for production usage where fault tolerance in the Predix Logging service is required. These planscan also be used to accelerate read access to data structures as there are two read slaves in thetopology.

Details of each plan type follow.

Predix Logging Dedicated Single-Node Configuration

Plan Feature Feature Description

HA Not included; the service will be interrupted upon any host reboot.

Persistence ElasticSearch indices are writte to local storage.

RDB Snapshots are enabled by default. The database will be savedwhen one of the following parameters has been reached:

• After 900 sec (15 min) if at least 1 key changed• After 300 sec (5 min) if at least 10 keys changed• After 60 sec if at least 10000 keys changed

Backups Hourly with 24-hour retention.

VM Dedicated VM with single Predix Logging process; no noisy neighbor issues.

© 2020 General Electric Company 1

Plan Feature Feature Description

Recommendation Recommended for advanced development or testing requirements, or in production

where Highly Available configuration is not required.

Predix Logging Dedicated Highly Available configuration

Plan Feature Feature Description

HA Single Read-Write Master replicating to two Read-Only Slaves; Redis \Sentinel

provides fast, automatic master promotion and client failover when using Sentinel-

aware clients. See the samples included in this document.

Persistence ElasticSearch indices are sharded, replicated, and stored both locally and on a second

node in the cluster.

RDB Snapshots are enabled by default. The database will be savedwhen one of the following parameters has been reached:

• After 900 sec (15 min) if at least 1 key changed• After 300 sec (5 min) if at least 10 keys changed• After 60 sec if at least 10000 keys changed

Backups Hourly with 24-hour retention

VM Dedicated VMs with single Predix Logging process each; clients can connect to slaves

to accelerate read performance.

Recommendation Good for production environments that require Highly Available configuration.

2 © 2020 General Electric Company

Getting Started with Predix Logging

Creating Predix Logging Service InstanceCreate a Predix Logging service instance to collect and mange log files.

About This Task

Use this process to determine if Predix Logging is available in the Cloud Foundry Marketplace, and tocreate your Predix Logging service instance.

Procedure

1. To retrieve the name of the Logging service, list the services in the Cloud Foundry marketplace.cf marketplaceThe service plans are listed:

'

service plans descriptionpredix-logging Dedicated-1-L20*, Dedicated-3HA-L40* Logging service for the Predix Platform

2. Creating a Predix Logging service instance and binding an App is all it takes to start using this service.

$ cf create-service predix-logging <plan-name> <your-service-instance-name>

Note: This step takes about 3 to 9 mins. Check status of service creation using cf servicescommand.

For example:

$ cf servicesGetting services in org xxx / space yyy as predix.user1@ge.com...OK

name service plan bound apps last operationmy-log-service predix-logging Dedicated-1-L20* create succeeded

3. Bind the Predix Logging service instance to your application.

cf bind-service <your-app-name> <your-service-instance-name>4. Restage your application.

cf restage <your-app-name>

© 2020 General Electric Company 3

Setting up the Predix Logging Kibana InterfaceView the application environment to retrieve the Kibana connection information for the Predix Loggingservice.

About This Task

Procedure

1. View the application environment to get kibana connection information.$ cf env <your-app-name>

2. Point a browser to the kibana_url and log in with the username/password from the previous step. Hereis an example ENV:

"kibana_url": "https://kibana-8c667b6c.run.aws.ice.predix.io","password": "sample_password","username": "sample_user"

Note:

When you access Kibana for the first time, you will be required to "Configure an index pattern". Followthese guidelines to define the index:

• Index contains time-based events - checked (default)• Use event times to create index names [DEPRECATED] - unchecked (default)• Index name or pattern: logstash-* - (default)• Index name or pattern: logstash-* - (default)• Time-field name - select @timestamp

◦ If @timestamp is not available as an option then it means there are no logs flowing into theservice instance

◦ Bind an app to the new service instance and refresh Kibana to see if @timestamp is available inthis drop-down menu

◦ Fewer than than 5 minutes will pass from the time an app is bound to Kibana seeing@timestamp in the Time-field name drop-down menu

• Click on Create

4 © 2020 General Electric Company

Using Predix Logging

Data Persistence and Back for Predix LoggingLearn how to manage your data and perform backups for Predix Logging service.

How Data is Persisted for Single-Node Configurations in Predix Logging

• Elastic Search indices are written to local storage.• If a single-node plan is rebooted, or the machine needs to be re-provisioned, data will be recovered

from the disk.

Important: The Predix Logging service supports a maximum of 5000 Elastic Search indices.

How Data is Persisted for Highly Aailable Configurations

• Elastic Search indices are sharded, replicated and stored both locally and on a second node in thecluster.

• If a data volume becomes corrupted, contact Predix support to initiate a restore from backup.

Important: The Predix Logging service supports a maximum of 5000 Elastic Search indices.

What is Backed Up in Predix Logging

A snapshot of the data volume occurs every hour and will be retained for 24 hours.

There is no self-service interface for downloading or restoring backups. You must contact Predix Supportto initiate a restore from a recent backup.

Important: By default, the Predix Logging service retains the log data for seven days.

Migrating from an Existing Service or Plan to a New Service orPlan

Questions about migrating to a new service or plan. Find the answers here.

How do I change plans from one to another?

• Create a new instance with the desired plan type.• Unbind the old service instance from your application.• Bind the new service instance to your applications, and restage the app.

Is there a way I can migrate my data to a new service instance?Not at this time.

Troubleshooting Predix LoggingFind answers to common questions for creating the Predix Logging service instance.

Service Instance - Create Service is in progress for a long time.The service instance creation takes about 3 to 9 minutes. If it takes over 20 minutes and if the statusis still in-progress, create a new service instance.

© 2020 General Electric Company 5

Service Instance - Create failed.Create a new service instance.

6 © 2020 General Electric Company