You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Alerting rules allow you to define alert conditions based on Prometheus expression language expressions and to send notifications about firing alerts to an external service. Whenever the alert expression results in one or more vector elements at a given point in time, the alert counts as active for these elements' label sets. If you have installed Prometheus with Helm, you have a Custom Resource Definitions (CRD) called PrometheusRule. Below is an example

PrometheusRule.yaml 
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  annotations:
    meta.helm.sh/release-name: mon
    meta.helm.sh/release-namespace: monitoring
  labels:
    app: kube-prometheus-stack
    release: mon
  name: regolatest
  namespace: monitoring
spec:
  groups:
  - name: customRules
    rules:
    - alert: rateGraph
      expr: rate(prometheus_http_requests_total{handler="/graph"}[5m]) * 60 >= 2
      labels:
        severity: none
      annotations:
        description: This alert is activated if, in the last 5 minutes, the http requests per minute towards the sub-path "/graph" of the Prometheus dashboard exceeds the value 2
        summary: Many requests to "graph"
    - alert: rateAlerts
      expr: rate(prometheus_http_requests_total{handler="/alerts"}[5m]) * 60 >= 3
      labels:
        severity: none
      annotations:
        description: This alert is activated if, in the last 5 minutes, the http requests per minute towards the sub-path "/alerts" of the Prometheus dashboard exceeds the value 3
        summary: Many requests to "alerts"

In this example we have inserted 2 rules that evaluate the rate of http requests to Prometheus dashboard pages. We will then update these web pages via the browser, to trigger the alert. Also note the labels adopted in this manifest.

Prometheus CRD 
# This is just an excerpt from the output of the "describe" command
$ kubectl describe -n monitoring prometheus 
spec:
  Rule Selector:
    Match Labels:
      App:      kube-prometheus-stack
      Release:  mon
  • No labels