...
| Code Block |
|---|
| language | bash |
|---|
| title | .env |
|---|
| collapse | true |
|---|
|
COMPOSE_PROJECT_NAME=es
CERTS_DIR=/usr/share/elasticsearch/config/certificates
VERSION=7.16.13 |
| Code Block |
|---|
| language | yml |
|---|
| title | create-certs.yml |
|---|
| collapse | true |
|---|
|
version: '2.2'
services:
create_certs:
image: docker.elastic.co/elasticsearch/elasticsearch:${VERSION}
container_name: create_certs
command: >
bash -c '
yum install -y -q -e 0 unzip;
if [[ ! -f /certs/bundle.zip ]]; then
bin/elasticsearch-certutil cert --silent --pem --in config/certificates/instances.yml -out /certs/bundle.zip;
unzip /certs/bundle.zip -d /certs;
fi;
chown -R 1000:0 /certs
'
working_dir: /usr/share/elasticsearch
volumes:
- certs:/certs
- .:/usr/share/elasticsearch/config/certificates
networks:
- elastic
volumes:
certs:
driver: local
networks:
elastic:
driver: bridge |
...
| Code Block |
|---|
| language | yml |
|---|
| title | elastic-docker-tls.yml |
|---|
| collapse | true |
|---|
|
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:${VERSION}
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.type=single-node
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.license.self_generated.type=trial # Generate and apply a trial license that supports TLS
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true # Enable TLS to encrypt client communications
- xpack.security.http.ssl.key=$CERTS_DIR/es01/es01.key
- xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
- xpack.security.http.ssl.certificate=$CERTS_DIR/es01/es01.crt
- xpack.security.transport.ssl.enabled=true # Enable TLS to encrypt internode communications
- xpack.security.transport.ssl.verification_mode=certificate # Allow the use of self-signed certificates by not requiring hostname verification
- xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
- xpack.security.transport.ssl.certificate=$CERTS_DIR/es01/es01.crt
- xpack.security.transport.ssl.key=$CERTS_DIR/es01/es01.key
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
- certs:$CERTS_DIR
ports:
- 9200:9200
networks:
- elastic
healthcheck:
test: curl --cacert $CERTS_DIR/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
interval: 30s
timeout: 10s
retries: 5
kib01:
image: docker.elastic.co/kibana/kibana:${VERSION}
container_name: kib01
depends_on: {"es01": {"condition": "service_healthy"}}
ports:
- 5601443:5601 # <--- Pay attention !!!
environment:
SERVERNAME: localhost
ELASTICSEARCH_URL: https://es01:9200
ELASTICSEARCH_HOSTS: https://es01:9200
ELASTICSEARCH_USERNAME: kibana_system
ELASTICSEARCH_PASSWORD: CHANGEME # <--- Pay attention !!!
ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES: $CERTS_DIR/ca/ca.crt
SERVER_SSL_ENABLED: "true"
SERVER_SSL_KEY: $CERTS_DIR/kib01/kib01.key
SERVER_SSL_CERTIFICATE: $CERTS_DIR/kib01/kib01.crt
volumes:
- certs:$CERTS_DIR
networks:
- elastic
volumes:
data01:
driver: local
certs:
driver: local
networks:
elastic:
driver: bridge |
...
Make a note of the generated passwords. You must configure the kibana_system user password in the compose file to enable Kibana to connect to ElasticsearchElasticSearch, and you’ll need the password for the elastic superuser to log in to Kibana and submit requests to ElasticsearchElasticSearch. Therefore, set ELASTICSEARCH_PASSWORD in the elastic-docker-tls.yml compose file to the password generated for the kibana_system user.
...
| Code Block |
|---|
| language | bash |
|---|
| title | Restart ESK |
|---|
| collapse | true |
|---|
|
$ docker-compose -f elastic-docker-tls.yml stop
Stopping kib01 ... done
Stopping es01 ... done
$ docker-compose -f elastic-docker-tls.yml up -d
Starting es01 ... done
Recreating kib01 ... done |
After the restart, now we can connect to the Kibana dashboard, using the elastic user credentials, and verify that the service is working (remember to use the https protocol). As already seen, you can tear down the containers, network, and volumes by running running docker-compose -f elastic-docker-tls.yml down -v.
Run FileBeat on K8s (TLS enabled)
As seen in the parent paragraph, now we have to take care of sending the logs from the K8s cluster to ESK. The procedure is similar to the one already seen, with some small differences. After downloading the filebeat-kubernetes.yaml, make the following changes
| Code Block |
|---|
| language | yml |
|---|
| title | filebeat-kubernetes.yaml |
|---|
| collapse | true |
|---|
|
output.elasticsearch:
hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
protocol: https # <--- Pay attention !!!
username: ${ELASTICSEARCH_USERNAME}
password: ${ELASTICSEARCH_PASSWORD}
ssl:
certificate_authorities: |
-----BEGIN CERTIFICATE-----
MIIDSTCCAjGgAwIBAgIUDVhtJ8gwO3EZJpY3/OOpv0FwAu0wDQYJKoZIhvcNAQEL
.
.
a7fWt1++yAAS3OHknl7MV8XlnO/S4a3nJz4AH/o=
-----END CERTIFICATE-----
certificate: |
-----BEGIN CERTIFICATE-----
MIIDQDCCAiigAwIBAgIUTy7I4LGEsEhuY69EqGN61v6cBqEwDQYJKoZIhvcNAQEL
.
.
apL+xAsTg06BunbXKYBVZnxIxTA=
-----END CERTIFICATE-----
key: |
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAtPRUO3tAvzWnyZR81qLMbHiWIOnv4lf4fEhSL0L/Oa1XvB+O
.
.
sMpqfmyHtjnNe9VV4u52NrKIDfcpHigXSAGAf2w/LYm/oPsksRCD
-----END RSA PRIVATE KEY-----
-----------------------------------------------------
env:
- name: ELASTICSEARCH_HOST
value: <IP> # <--- Enter the Floating/Internal IP of the host with elasticsearch
- name: ELASTICSEARCH_PORT
value: "9200"
- name: ELASTICSEARCH_USERNAME
value: elastic
- name: ELASTICSEARCH_PASSWORD
value: <elastic_user_password> |
The certificates are located inside the es01 container, in the path indicated in the .env file. Retrieve the ca.crt, es01.crt and es01.key files, present in the ca and es01 folder, respectively.
| Code Block |
|---|
| language | bash |
|---|
| title | Retrieve the certificates |
|---|
| collapse | true |
|---|
|
# To enter the container
$ docker exec -it es01 /bin/bash
# Path of the certificates inside the container
root@0d928f8ebff2:/usr/share/elasticsearch/config/certificates# ls
bundle.zip ca es01 kib01 |