Page History

...

# This is just an excerpt from the output of the "describe" command
$ kubectl describe -n monitoring prometheus 
specSpec:
  Rule Selector:
    Match Labels:
      App:      kube-prometheus-stack
      Release:  mon

...

Now let's move to the Prometheus dashboard and update the sub-path /graph and /alerts (framed in yellow at the top left of the image) a few times (15-20 times should be enough), in order to activate the rules adopted. As you can see from the image, the created alerts have been activated and, in fact, are in firing status. After a few minutes, if no other http requests are made, these alerts will return to the starting status, i.e. inactive. The Watchdog alert, at the top of the screenshot, is always present: is an alert meant to ensure that the entire alerting pipeline is functional.Firing alerts

Send alert via e-mail

...

Let's try to send our alerts by e-mail. To do this we will use 2 other CRDs: Alertmanager and AlertmanagerConfig. Note the analogy with Prometheus and PrometheusRule. Also in this case we have linked the 2 CRDs through labels (however they can be chosen arbitrarily). Launch a describe on the Alertmanager component

Code Block
language bash collapse true
# This is just an excerpt from the output of the "describe" command $ kubectl describe -n monitoring alertmanager Spec: Alertmanager Config Selector: Match Labels: Alertmanager: config

Create and configure the AlertmanagerConfig component for sending e-mails. As you can see, Gmail is used as the mail server here. First of all it is advisable to generate a password dedicated to this purpose and insert it in the Secret below, after having encrypted it with the command echo <password> | base64 -w0.

Code Block
language yml title AlertmanagerConfig.yaml collapse true
apiVersion: monitoring.coreos.com/v1alpha1 kind: AlertmanagerConfig metadata: name: alert-email namespace: monitoring labels: alertmanager: config spec: route: groupBy: [severity] receiver: 'notifications' groupWait: 30s groupInterval: 5m repeatInterval: 12h receivers: - name: 'notifications' emailConfigs: - to: <to@example.com> from: <from@gmail.com> smarthost: smtp.gmail.com:587 authUsername: <from@gmail.com> authIdentity: <from@gmail.com> authPassword: name: gmail-pass key: alertmanager.yaml sendResolved: true headers: - key: From value: <from@gmail.com> - key: Subject value: 'Alertmanager notification' - key: To value: to@example<to@example.comcom> --- apiVersion: v1 kind: Secret type: Opaque metadata: name: gmail-pass namespace: monitoring data: alertmanager.yaml: <pass_encode_base64>

...