Matthias Runge
CentOS Opstools SIG - Update
February 1, 2019
1
CentOS Opstools SIG Update
● Intro● What is included● A closer look at performance monitoring● Taking the next step
Agenda
2
INTRODUCTION
3
CentOS Opstools SIG Update4
● Intended to support operators in second day tasks● a piece in the puzzle for RHOSP
Opstools - tools for operators
CentOS Opstools SIG Update5
Community and products
**
CentOS Opstools SIG Update6
Relationship with community and OSP repos
collectdOpenStack
CentOS opstools
CentOS MessagingRDO
QPID
Cloud SIG
Red Hat AMQ
oVirt
OSP OSP opstools
RHV
EPEL
CENTRALIZED LOGGING
7
CentOS Opstools SIG Update8
Centralized logging
fluentd fluentd
fluentd
fluentd
elasticsearch
AVAILABILITY MONITORING
9
CentOS Opstools SIG Update10
Availability monitoring - Sensu
PERFORMANCE MONITORING
11
CentOS Opstools SIG Update
● collectd● Graphite/CarbonDB● Grafana● Gnocchi
12
Performance monitoring (past)
collectd
collectd
collectd
graphite
GrafanaMonitoring node
CentOS Opstools SIG Update
● collectd 5.8.0, released Nov 17, 2017● release 5.8.1 in preparation● small resource usage● around since 2005● about 80 plugins, reporting mostly low level system information
○ bindings for Python, Java, exec, Perl● generates both metrics and events
13
collectd
CentOS Opstools SIG Update14
Performance monitoring (past) - HA
collectd
collectd
collectd
graphite
GrafanaMonitoring node
CentOS Opstools SIG Update
Kubernetes is a portable, extensible open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation.
15
Kubernetes
CentOS Opstools SIG Update16
Prometheus
CentOS Opstools SIG Update
● Decouple Prometheus instances from configuration
17
Prometheus operator
CentOS Opstools SIG Update
● QPID proton● dispatch router● qpid cpp broker
18
Apache QPID
qdr qdr qdrqdr
producer consumer
broker broker
CentOS Opstools SIG Update
infra node
19
Service assurance framework
collectd
qdr
qdr
qdr
qdr
Prometheus
grafana
telemetry consumer
kubernetes cluster
broker broker
CentOS Opstools SIG Update20
Grafana
CentOS Opstools SIG Update21
● Alarming rules defined in prometheus● alarm actions executed by alertmanager● - name: openstack-api slow
rules: - alert: openstack-api expr: avg_over_time(collectd_curl_response_time[30s]) > 0.2 for: 3m labels: severity: pager annotations: summary: openstack api slow
Alarming
OUTLOOK
22
CentOS Opstools SIG Update
● integration of “self healing” parts?● adding tools like vitrage, or doctor?● deprecating sensu in favor of alertmanager
23
Outlook
THANK YOU!
plus.google.com/+RedHat
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHatNews
25