Docker Compose YAML Elasticsearch HTTPS

https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls-docker.html

---------------------------------

services:

  create_certs:

    container_name: create_certs

    image: docker.elastic.co/elasticsearch/elasticsearch:7.13.0

    # command: >

    #   bash -c '

    #     if [[ ! -f /certs/bundle.zip ]]; then

    #       bin/elasticsearch-certutil cert --silent --pem --in config/certificates/instances.yml -out /certs/bundle.zip;

    #       unzip /certs/bundle.zip -d /certs; 

    #     fi;

    #     chown -R 1000:0 /certs

    #   '

    # user: "0"

    # working_dir: /usr/share/elasticsearch

    # volumes: ['certs:/certs', '.:/usr/share/elasticsearch/config/certificates']

    command: >

      bash -c '

        if [[ ! -f ./config/certificates/elastic-certificates.p12 ]]; then

          bin/elasticsearch-certutil cert -out config/certificates/elastic-certificates.p12 -pass ""

        fi;

        chown -R 1000:0 /usr/share/elasticsearch/config/certificates

      '

    user: "0"

    working_dir: /usr/share/elasticsearch

    volumes: ['certs:/usr/share/elasticsearch/config/certificates']

  elasticsearch:

    container_name: elasticsearch

    depends_on: [create_certs]

    image: docker.elastic.co/elasticsearch/elasticsearch:7.13.0

    environment:

      - cluster.name=docker-cluster

      - discovery.type=single-node

      - bootstrap.memory_lock=true

      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

      - ELASTIC_PASSWORD=$ELASTIC_PASSWORD # password for default user: elastic 

      - xpack.security.enabled=true

      - xpack.security.transport.ssl.enabled=true

      - xpack.security.transport.ssl.verification_mode=certificate

      - xpack.security.transport.ssl.keystore.path=$CERTS_DIR/elastic-certificates.p12

      - xpack.security.transport.ssl.truststore.path=$CERTS_DIR/elastic-certificates.p12

      - xpack.security.http.ssl.enabled=true

      - xpack.security.http.ssl.verification_mode=none

      - xpack.security.http.ssl.truststore.path=$CERTS_DIR/elastic-certificates.p12

      - xpack.security.http.ssl.keystore.path=$CERTS_DIR/elastic-certificates.p12

      # - xpack.license.self_generated.type=trial 

      # - xpack.security.enabled=true

      # - xpack.security.http.ssl.enabled=true

      # - xpack.security.http.ssl.key=$CERTS_DIR/es01/es01.key

      # - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt

      # - xpack.security.http.ssl.certificate=$CERTS_DIR/es01/es01.crt

      # - xpack.security.transport.ssl.enabled=true

      # - xpack.security.transport.ssl.verification_mode=certificate 

      # - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt

      # - xpack.security.transport.ssl.certificate=$CERTS_DIR/es01/es01.crt

      # - xpack.security.transport.ssl.key=$CERTS_DIR/es01/es01.key

    volumes: ['esdata:/usr/share/elasticsearch/data', 'certs:$CERTS_DIR']

    ulimits:

      nofile:

        soft: 65536

        hard: 65536

      memlock:

        soft: -1

        hard: -1

    ports:

      - "9200:9200"

volumes: {"esdata", "certs"}

TLS SSL Docker Elasticsearch

https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup.html#generate-certificates

https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup-https.html#encrypt-http-communication

--------------------------------------------------------------------------------------------------------

https://stackoverflow.com/questions/50832249/enable-authentication-in-elasticsearch-with-docker-environment-variable

https://dev.to/thehoodsdev/securing-our-dockerized-elastic-stack-3o15

https://medium.com/@mandeep_m91/setting-up-elasticsearch-and-kibana-on-docker-with-x-pack-security-enabled-6875b63902e6

https://askubuntu.com/questions/772050/reset-the-password-in-ubuntu-linux-bash-in-windows

wsl --user root

elasticsearch has own cert creation module

https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#tls-http

xpack.security.enabled to true

OS

Basic 

Gold

Platinum

https://stackoverflow.com/questions/51445846/elasticsearch-max-virtual-memory-areas-vm-max-map-count-65530-is-too-low-inc/51447991#51447991

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

sudo sysctl -w vm.max_map_count=262144

wsl --user root

https://stackoverflow.com/questions/22049212/docker-copying-files-from-docker-container-to-host

docker cp <containerId>:/file/path/within/container /host/path/target

docker cp elasticsearch:/usr/share/elasticsearch  ~

docker run -it --rm --privileged --pid=host justincormack/nsenter1

PKCS#12 format

A PFX file indicates a certificate in PKCS#12 format

https://www.sslmarket.com/ssl/how-to-create-an-pfx-file#:~:text=A%20PFX%20file%20indicates%20a,need%20to%20deploy%20a%20certificate.

docker-compose down -v

https://www.bleepingcomputer.com/news/security/new-meow-attack-has-deleted-almost-4-000-unsecured-databases/

http://localhost:9200/_xpack

http://localhost:9200/

xpack.security.http.ssl.enabled

xpack.security.transport.ssl.enabled

 - xpack.security.transport.ssl.enabled=true

Docker Desktop + WSL2 + Ubuntu + WSL +docker

 docker run -it --rm --privileged --pid=host justincormack/nsenter1

\\wsl2$ - Special Shared Path on Windows - which hides complex Path of Windows 
Docker Desktop + Ubuntu(WSL2) --- linked
Docker commands run from both
but Images go in Docker Desktop only -  \\wsl$\docker-desktop-data\version-pack-data\community\docker
Volume should be uploaded from Ubuntu/WSL2
Volume upload from Windows wont work
You can upload Folders from Windows to \\wsl$ shared path -specfically here  - \\wsl$\Ubuntu-20.04\home\karankaw
and it will show up as ~ in WSL2/Ubuntu

Docker Logs

 docker logs  <web_container_ID_Or_Name>

 docker logs --follow web

Windows: Docker : exec :Docker Desktop

https://www.bretfisher.com/getting-a-shell-in-the-docker-for-windows-vm/

https://gist.github.com/BretFisher/ef4070b60ca7380846ff62f3cebd7856

Getting a Shell in the Docker for Windows Moby VM

Moby VM - Mean Dummy Linux VM running on WSL Windows

docker run -it --rm --privileged --pid=host justincormack/nsenter1

Docker Volumes - File Locations for docker which Persist after container

https://www.freecodecamp.org/news/where-are-docker-images-stored-docker-container-paths-explained/

https://stackoverflow.com/questions/34809646/what-is-the-purpose-of-volume-in-dockerfile/34810191#34810191

https://www.docker.com/blog/how-to-use-the-official-nginx-docker-image/   [Example]

Docker Volumes

It is possible to add a persistent store to containers to keep data longer than the container exists or to share the volume with the host or with other containers. A container can be started with a volume by using the -v option:

$ docker run --name nginx_container -v /var/log nginx

We can get information about the connected volume location by:
$ docker inspect nginx_container 

Adding Custom HTML

By default, Nginx looks in the /usr/share/nginx/html directory inside of the container for files to serve. We need to get our html files into this directory. A fairly simple way to do this is use a mounted volume. With mounted volumes, we are able to link a directory on our local machine and map that directory into our running container.

docker run -it --rm -d -p 8080:80 --name web -v ~/site-content:/usr/share/nginx/html nginx

docker    run   -v    /path/to/host/directory:/path/inside/the/container    image

When a docker container is deleted,
volume is not deleted by itself, atleast not by default.

Clean up space used by Docker
It is recommended to use the Docker command to clean up unused containers. Container, networks, images, and the build cache can be cleaned up by executing:
$ docker system prune -a
Additionally, you can also remove unused volumes by executing:
$ docker volumes prune

Docker Images : Where are they stored

https://www.freecodecamp.org/news/where-are-docker-images-stored-docker-container-paths-explained/

https://stackoverflow.com/questions/43181654/locating-data-volumes-in-docker-desktop-windows/64418064#64418064

Docker Desktop + WSL(Ubuntu-20)

\\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\

\\wsl$\docker-desktop-data\version-pack-data\community\docker\overlay2

Docker images

The heaviest contents are usually images. If you use the default storage driver overlay2, then your Docker images are stored in 

/var/lib/docker/overlay2  - List of all images 




/var/lib/docker/image/overlay2/imagedb/content/sha256 List of kinda Top Images




docker run -it --rm --privileged --pid=host justincormack/nsenter1
\\wsl2$ - Special Shared Path on Windows - which hides complex Path of Windows 
Docker Desktop + Ubuntu(WSL2) --- linked
Docker commands run from both
but Images go in Docker Desktop only -  \\wsl$\docker-desktop-data\version-pack-data\community\docker
Volume should be uploaded from Ubuntu/WSL2
Volume upload from Windows wont work
You can upload Folders from Windows to \\wsl$ shared path -specfically here  - \\wsl$\Ubuntu-20.04\home\karankaw
and it will show up as ~ in WSL2/Ubuntu