Quick Start Guide

A Grafana/Prometheus/Influxdb  installation has been set up with the purpose to monitor the DAFNE Control System (DCS).

...

Fig. 4 - Example of dashboard

Configuration Guide

Having different systems to monitor, with different OSes (Solaris 9, Linux Centos 3-8, Windows), is needed to found a performance tool/agent or methodology to monitoring the devices.

Prometheus "collector"

The recent linux OSes, as CentOS 6-8, or Windows could use prometheus for store the data of metrics. Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Prometheus fundamentally stores all data as time series: streams of timestamped values belonging to the same metric and the same set of labeled dimensions. Besides stored time series, Prometheus may generate temporary derived time series as the result of queries. (see https://prometheus.io/docs/introduction/overview/)

CentOS 6-8

 For CentOS 6-8 exists "node-exporter" agent. (see the page https://prometheus.io/docs/guides/node-exporter/). The guide show, the agent is possible installed that manually, but with current infrastructure of "configuration management" (Foreman/Puppet/Ansible) is possible automating the process by enable the specific module.   

...

 prometheus::node_exporter:
    proxy_server: http://squid.lnf.infn.it:3128
    service_enable: true

Windows

For Windows exists a Prometheus Agent similar to node_exporter for Linux calling  windows_exporter (see the page https://github.com/prometheus-community/windows_exporter/releases).

...

Prometheus configuration

To read machine metrics, Prometheus must be enabled. The Foreman machine is ready to do so. Below the configuration in foreman/puppet:

...

Influxdb "collector"

Node_exporter is not compatible with older operating systems, such as CentOS 3-5 or Solaris 9. In this case, it was decided to use a "call" curl to send data metrics within influxdb. This method is reliable and is very easy to implement in several old systems or on raw devices where only the curl command is present. Influxdb, similary to Prometheus, is the open source time series database that is part of the TICK (Telegraf, InfluxDB, Chronograf, Kapacitor) stack. It is designed to handle high write and query loads and provides a SQL-like query language called InfluxQL for interacting with data.

...

CentOS 3: https://baltig.infn.it/chaos-lnf-control/metrics.sh-centos3

Solaris 9

Install metrics.sh on Solaris 9 (see: https://baltig.infn.it/chaos-lnf-control/metrics.sh-solaris9):

...

# Install metrics.sh at /opt/metrics.sh
$ mkdir /opt; cd /opt
$ git clone https://baltig.infn.it/chaos-lnf-control/metrics.sh-solaris9.git metrics.sh
$ cd metrics.sh
# Without git, git clone into other machine and copy directory in solaris 9 machine
# Install the service
$ ln $PWD/init.d/metrics.sh /etc/init.d/metrics.sh
$ ln /etc/init.d/metrics.sh /etc/rc3.d/Smetrics.sh
$ ln /etc/init.d/metrics.sh /etc/rc0.d/Kmetrics.sh
# Copy config file
$ mkdir /etc/metrics.sh;
$ cp /opt/metrics.sh/metrics.ini /etc/metrics.sh/metrics.ini
# At this point you should edit your config file at
# /etc/metrics.sh/metrics.ini

# Start service
$ /etc/init.d/metrics.sh start

# If run with the default configuration where reporter is 'stdout', metrics
# will be written to /var/log/metrics.sh.log. Be aware that this file will
# grow fast.
$ tail -f /var/log/metrics.sh.log

# Stop service
$ /etc/init.d/metrics.sh stop

# Check service status
$ /etc/init.d/metrics.sh status

CentOS 3:

Install metrics.sh on CentOS 3:

# Install metrics.sh at /opt/metrics.sh
$ mkdir /opt; cd /opt
$ git clone https://baltig.infn.it/chaos-lnf-control/metrics.sh-solaris9.git metrics
$ cd metrics.sh
# Install the service
$ ln -s $PWD/init.d/metrics.sh /etc/init.d/metrics.sh
# Create a config file
$ mkdir /etc/metrics.sh && chmod 600 /etc/metrics.sh
$ ./metrics.sh -C > /etc/metrics.sh/metrics.ini
# At this point you should edit your config file at
# /etc/metrics.sh/metrics.ini

# Start service
$ service metrics.sh start

# If run with the default configuration where reporter is 'stdout', metrics
# will be written to /var/log/metrics.sh.log. Be aware that this file will
# grow fast.
$ tail -f /var/log/metrics.sh.log

# Stop service
$ service metrics.sh stop

# Check service status
$ service metrics.sh status

Creation DB into influxdb:

curl -i -XPOST http://vldantedbn001.lnf.infn.it:8086/query --data-urlencode "q=CREATE DATABASE metrics"

Customized file.ini to write data into influxdb:

....... 
[reporter influxdb]
;Send data to InfluxDB.
INFLUXDB_API_ENDPOINT=vldantedbn001.lnf.infn.it:8086/write?db=metrics
INFLUXDB_SEND_HOSTNAME=true
.......

...

GRAFANA:

Before to import the dashboard, yum must configure the Data Source, where is store the metrics. In this case, we need of two Data Sources: 

...