Kibana + timelion: time series with the elastic stack

1

Sylvain Wallez @bluxte

Kibana + Timelion: Time series with the Elastic Stack

2

Agenda

From ELK to Elastic Stack 5.0

Kibana

Timelion

Conclusion

1

2

3

4

3

4

The Elastic Stack

Elastic Cloud

Security

Monitoring

Aler0ng

Graph

X-Pack

Kibana User Interface

Elasticsearch Store, Index, & Analyze

Ingest Logstash Beats

+

5

What’s new in Elastic Stack 5.0 ?

‒  Dimensional fields – speed up and better compression of numerical data ‒  Ingest node – avoids using Logstash for simple setups ‒  Rollover & Shrink API – better handling of non-hot data ‒  Painless – new fast & secure scripting langage

‒  Complete UI redesign ‒  Control center: config, monitoring, dev tools… and dashboards ‒  Timelion: for time series

Unified release, same version number for all products

6

What’s new in Elastic Stack 5.0 ?

•  ‒  Up to 50% faster ‒  Integration with Kafka ‒  New filters

•  ‒  Libbeat – framework pour construire des agents ‒  MetricBeat, PacketBeat, FileBeat ‒  {Community}Beats, lots of them

‒  Immediate availability of new versions ‒  Automated upgrades

Unified release, same version number for all products

7

Agenda

From ELK to Elastic Stack 5.0

Kibana

Timelion

Conclusion

1

2

3

4

8

Kibana evolution

Data Visualization Management

9

Kibana evolution: 4.x

Data Visualization Management

Discover

Dashboard

Visualize

10

Kibana evolution: 4.x

Data Visualization Management

Discover

Dashboard

Visualize

Monitoring

11

Graph

Kibana evolution: 4.x

Data Visualization Management

Discover

Dashboard

Visualize

Monitoring

12

Graph

Timelion Sense

Kibana evolution: 4.x

Data Visualization Management

Discover

Dashboard

Visualize

Monitoring

13

Discover

Dashboard

Graph DevTools

Timelion Console

Monitoring

Visualize

Kibana evolution: 5.0

Data Visualization Management

14

Discover

Dashboard

Graph

Settings

Users

DevTools

Timelion

Monitoring

Visualize

Console

Kibana evolution: 5.0

Data Visualization Management

15

Creating a Kibana dashboard

1 2 3 4 Configure

Select indices

Discover

Explore & filter

Visualize

Create charts

Dashboard

Layout charts

5 It’s alive!

Live update &

filtering

16

17

Creating a Kibana dashboard

1 2 3 4 Configure

Select indices

Discover

Explore & filter

Visualize

Create charts

Dashboard

Layout charts

5 It’s alive!

Live update &

filtering

18

19

Creating a Kibana dashboard

1 2 3 4 Configure

Select indices

Discover

Explore & filter

Visualize

Create charts

Dashboard

Layout charts

5 It’s alive!

Live update &

filtering

20

21

Creating a Kibana dashboard

1 2 3 4 Configure

Select indices

Discover

Explore & filter

Visualize

Create charts

Dashboard

Layout charts

5 It’s alive!

Live update &

filtering

22

23

Creating a Kibana dashboard

1 2 3 4 Configure

Select indices

Discover

Explore & filter

Visualize

Create charts

Dashboard

Layout charts

5 It’s alive!

Live update &

filtering

24

25 25

Demo!

26

Agenda

From ELK to Elastic Stack 5.0

Kibana

Timelion

Conclusion

1

2

3

4

27

Why Timelion?

• Elasticsearch queries ‒  Select lots of items (lightning fast) ‒  Aggregations make them understandable

• Need more than that ‒  Correlation, calculation, filtering ‒  Assemble multiple data sources

• Timelion started as an experiment ‒  Escape the UI and use the flexibility of a language

Because there’s more than search

28

Time series resampling A common time reference to allow calculations

Value

Value

Time

Time

Bucket

29

Timelion expression language A transformation & aggregation pipeline

ES

World Bank

Graphite

- Sampling - Aggregation

- Calculation - Graphic attributes

Data sources

Transformations

Time series - Data - Metadata - Graphic attributes

30

The Timelion expression language

• Functions ‒  abs, cusum, derivative, fit, holt, log, min, max, movingaverage, movingstd, sum, trend

• Combinations ‒  add, divide, multiply, plus, substract

• Filtering ‒  condition, if

• Graphic attributes ‒  bars, color, hide, label, legend, lines, points, range, title, yaxis

• Data sources ‒  elastic, graphite, wbi, quantl

A rich function library

31

The Timelion expression language

32 32

Demo!

33

Agenda

From ELK to Elastic Stack 5.0

Kibana

Timelion

Conclusion

1

2

3

4

34

Questions?

Answers!