You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Installation with Docker-compose (TLS enabled)

To get an ESK up and running in Docker with security enabled, create the following compose and configuration files:

  • instances.yml identifies the instances you need to create certificates for;
  • .env sets environment variables to specify the ElasticSearch version and the location where the ElasticSearch certificates will be created;
  • create-certs.yml is a Docker-compose file that launches a container to generate the certificates for ESK;
  • elastic-docker-tls.yml is a Docker-compose file that brings up a ESK instance with Transport Layer Security (TLS) enabled.
instances.yml 
instances:
  - name: es01
    dns:
      - es01
      - localhost
    ip:
      - 127.0.0.1
  - name: kib01
    dns:
      - kib01
      - localhost
.env 
COMPOSE_PROJECT_NAME=es
CERTS_DIR=/usr/share/elasticsearch/config/certificates
VERSION=7.16.1
create-certs.yml 
version: '2.2'
services:
  create_certs:
    image: docker.elastic.co/elasticsearch/elasticsearch:${VERSION}
    container_name: create_certs
    command: >
      bash -c '
        yum install -y -q -e 0 unzip;
        if [[ ! -f /certs/bundle.zip ]]; then
          bin/elasticsearch-certutil cert --silent --pem --in config/certificates/instances.yml -out /certs/bundle.zip;
          unzip /certs/bundle.zip -d /certs;
        fi;
        chown -R 1000:0 /certs
      '
    working_dir: /usr/share/elasticsearch
    volumes:
      - certs:/certs
      - .:/usr/share/elasticsearch/config/certificates
    networks:
      - elastic
volumes:
  certs:
    driver: local
networks:
  elastic:
    driver: bridge
elastic-docker-tls.yml 
version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:${VERSION}
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.license.self_generated.type=trial # Generate and apply a trial license that supports TLS
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true # Enable TLS to encrypt client communications
      - xpack.security.http.ssl.key=$CERTS_DIR/es01/es01.key
      - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.http.ssl.certificate=$CERTS_DIR/es01/es01.crt
      - xpack.security.transport.ssl.enabled=true # Enable TLS to encrypt internode communications
      - xpack.security.transport.ssl.verification_mode=certificate # Allow the use of self-signed certificates by not requiring hostname verification
      - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.transport.ssl.certificate=$CERTS_DIR/es01/es01.crt
      - xpack.security.transport.ssl.key=$CERTS_DIR/es01/es01.key
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
      - certs:$CERTS_DIR
    ports:
      - 9200:9200
    networks:
      - elastic
    healthcheck:
      test: curl --cacert $CERTS_DIR/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
      interval: 30s
      timeout: 10s
      retries: 5
  kib01:
    image: docker.elastic.co/kibana/kibana:${VERSION}
    container_name: kib01
    depends_on: {"es01": {"condition": "service_healthy"}}
    ports:
      - 5601:5601
    environment:
      SERVERNAME: localhost
      ELASTICSEARCH_URL: https://es01:9200
      ELASTICSEARCH_HOSTS: https://es01:9200
      ELASTICSEARCH_USERNAME: kibana_system
      ELASTICSEARCH_PASSWORD: CHANGEME # <--- Pay attention !!!
      ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES: $CERTS_DIR/ca/ca.crt
      SERVER_SSL_ENABLED: "true"
      SERVER_SSL_KEY: $CERTS_DIR/kib01/kib01.key
      SERVER_SSL_CERTIFICATE: $CERTS_DIR/kib01/kib01.crt
    volumes:
      - certs:$CERTS_DIR
    networks:
      - elastic
volumes:
  data01:
    driver: local
  certs:
    driver: local
networks:
  elastic:
    driver: bridge

Generate certificates for ElasticSearch by bringing up the create-certs container, then bring up the ESK

Generate certificates and start ESK 
$ docker-compose -f create-certs.yml run --rm create_certs
Creating es_create_certs_run ... done
Archive:  /certs/bundle.zip
   creating: /certs/ca/
  inflating: /certs/ca/ca.crt
   creating: /certs/es01/
  inflating: /certs/es01/es01.crt
  inflating: /certs/es01/es01.key
   creating: /certs/kib01/
  inflating: /certs/kib01/kib01.crt
  inflating: /certs/kib01/kib01.key

$ docker-compose -f elastic-docker-tls.yml up -d
Creating es01 ... done
Creating kib01 ... done

At this point, Kibana cannot connect to the Elasticsearch cluster. You must generate a password for the built-in kibana_system user, update the ELASTICSEARCH_PASSWORD in the compose file, and restart to enable Kibana to communicate with the secured cluster.

Run the elasticsearch-setup-passwords tool to generate passwords for all built-in users, including the kibana_system user

Generate passwords 
$ docker exec es01 /bin/bash -c "bin/elasticsearch-setup-passwords auto --batch --url https://es01:9200"
Changed password for user apm_system
PASSWORD apm_system = <Password>

Changed password for user kibana_system
PASSWORD kibana_system = <Password>

Changed password for user kibana
PASSWORD kibana = <Password>

Changed password for user logstash_system
PASSWORD logstash_system = <Password>

Changed password for user beats_system
PASSWORD beats_system = <Password>

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = <Password>

Changed password for user elastic
PASSWORD elastic = <Password>

Make a note of the generated passwords. You must configure the kibana_system user password in the compose file to enable Kibana to connect to ElasticSearch, and you’ll need the password for the elastic superuser to log in to Kibana and submit requests to ElasticSearch. Therefore, set ELASTICSEARCH_PASSWORD in the elastic-docker-tls.yml compose file to the password generated for the kibana_system user.

At this point, use docker-compose to restart the ESK

Restart ESK
$ docker-compose -f elastic-docker-tls.yml stop
Stopping kib01 ... done
Stopping es01  ... done

$ docker-compose -f elastic-docker-tls.yml up -d
Starting es01 ... done
Recreating kib01 ... done

After the restart, now we can connect to the Kibana dashboard, using the elastic user credentials, and verify that the service is working (remember to use the https protocol). As already seen, you can tear down the containers, network, and volumes by running docker-compose -f elastic-docker-tls.yml down -v.

Run FileBeat on K8s (TLS enabled)

  • No labels