Page History

...

Secret

To allow Prometheus to securely connect to etcd, we need a secret. To create a secret we use the following files, which should already be in our possession

$ kubectl -n monitoring create secret generic <secret_name> --from-file=/etc/kubernetes/pki/etcd/ca.crt --from-file=/etc/kubernetes/pki/apiserver-etcd-client.crt --from-file=/etc/kubernetes/pki/apiserver-etcd-client.key

The Now we have to insert the newly created secret in the spec of the Prometheus "component". In this way, the mentioned files will be mounted inside the prometheus-0 pod, in the path /etc/prometheus/secrets/<secret_name>. So

$ kubectl edit prometheus -n monitoring
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  annotations:
    meta.helm.sh/release-name: prometheus
    meta.helm.sh/release-namespace: monitoring
.
.
.
spec:
.
.
.
  ruleSelector:
    matchLabels:
      app: kube-prometheus-stack
      release: prometheus
  secrets:
  - <secret_name>	# <--- Insert secret here
  securityContext:
    fsGroup: 2000
    runAsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
.
.
.

Service (with endpoints)

Second, the service that will describe our etcd cluster must be created. Moreover, here were are going to list the endpoints for our etcd servers and then attach them to our service. Change the IP addresses to match the IPs of your etcd servers. The way these endpoints are connected to the service is through the name property of the metadata: this must match the name of the service.

...

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    k8s-app: etcd
    release: prometheus
  name: prometheus-etcd
  namespace: monitoring
spec:
  endpoints:
  - port: metrics
    interval: 30s
    scheme: https
    tlsConfig:
      ca_file: /etc/prometheus/secrets/prometheus-etcd<secret_name>/ca.crt
      cert_file: /etc/prometheus/secrets/prometheus-etcd<secret_name>/apiserver-etcd-client.crt
      key_file: /etc/prometheus/secrets/prometheus-etcd<secret_name>/apiserver-etcd-client.key
  jobLabel: k8s-app
  namespaceSelector:
    matchNames:
    - monitoring
  selector:
    matchLabels:
      k8s-app: etcd

That’s it. Now we just need to apply these files to our cluster.