Tuesday, Nov 14, 2023

docker

In my previous post, I discussed Docker Networks in the context of Compose, as you can find in "Configuring Docker Networks with Compose".

In this article, my focus shifts to exploring Docker storage and data persistence alternatives.

Default behavior

By default, Docker containers possess a writable ephemeral layer. Any files that are created or modified within the container are stored in this layer, provided that no bind-mounts or volumes have been connected to the container. However, when the container is terminated, both the files and the container itself are removed. While this ephemeral nature is suitable for certain containers and applications, it may not be appropriate for others, particularly when dealing with sensitive data such as a database.

In addition to this non-persistent behavior, the default writable layer also carry's a performance burden as well as increasing the size of the container itself.

Docker Bind mounts and Volumes

Docker provides two additional file storage and data persistence mechanisms for containers, Bind-mounts and Volumes.

There is an third mount option called, tmpfs, that I won't be expanding on in this post. TLDR; allows you to persist data in memory on the Host machine rather than writing to disc. After ollocated memory is freed on the host machine, the data is lost.

Regardless of mount type, the Docker container itself does not know the difference and is an abstraction as far as it is concerned.

Bind mounts

Bind mounts enable the inclusion of files and directories from the host machine into a container, serving various purposes, such as sharing configuration files. For instance, you can utilize bind mounts to share files like /etc/resolve.conf for DNS resolution or web application build artifacts generated by a development environment running on the host machine, as opposed to a Docker container.

An essential point to remember is that if the file or directory doesn't exist on the host machine, Docker will generate them as needed.

The Docker documentation advises prioritizing the use of Volumes over Bind mounts when feasible because Volumes offer a broader spectrum of functionality.

Volumes

Docker Volumes differ from Bind mounts in that Docker itself creates and oversees them. When running Docker on a Linux system, the file path for Docker Volumes is typically located at /var/lib/docker/volumes/.

Since Docker manages these files and data, it's essential that external processes not interfere with them in any manner. In situations where you need other applications to read or modify shared files on the host, a bind mount is a more suitable option.

Using Bind mounts and Volumes with the Docker cli

The command flag options for both Docker Volumes and Bind mounts are very similar. When creating and running Docker containers via the command line, both support options like --mount and --volumes (or -v).

In the case of --volumes, which comprises three fields separated by a colon :, the initial field is where you specify either the named volume for Volumes or the path to a file or directory on the host machine for Bind mounts.

In the case of --mount, it is comprised of several key-value pair tuples separated by commas ,, and the order of the keys is not important. These key-value pairs encompass attributes such as type, source, destination, and so on.

Docker recommends using the --mount over --volumes option, as it's easier to learn for new users.

Managing Volumes

You can create a Docker volume directly using the command line outside the context of a container by running the following command: docker volume create. This grants you greater control over the creation process, including specifying the driver type. Afterward, you can then mount this volume to one or multiple containers as needed.

Example, can create volume called, my-vol with

docker volume create my-vol

You can attach the previously created volume, named 'my-vol,' to a Docker container using the -v or --volumes flag option. It's important to mention that if the 'my-vol' volume doesn't already exist, Docker will automatically create the volume for you.

docker run -d \
--name devtest \
-v my-vol:/app \
nginx:latest

Alternatively, with the --mount flag option.

docker run -d \
--name devtest \
--mount source=my-vol,target=/app \
nginx:latest

Additional commands to interact with volumes:

• docker volume ls: To view available containers on the Docker host
• docker volume prune: To remove unused volumes, which is not removed automatically when the containers using the volume are removed.
• docker volume rm <volume>: To remove an individual volume by its name.
• docker volume inspect <volume>: To see additional volume configuration of a given volume.

Managing Bind mounts

Docker doesn't directly manage the individual files and directories on the host machine. You can only mount them to containers during container creation or execution. There are no specific command-line instructions outside of the container context for managing these files and directories.

To create a bind-mount with --volume or -v

docker run -d \
-it \
--name devtest \
-v "$(pwd)"/target:/app \
nginx:latest

When using --volume, If the file or directory on the host machine don't exist yet, Docker creates the endpoint for you.

Using --mount

docker run -d \
-it \
--name devtest \
--mount type=bind,source="$(pwd)"/target,target=/app \
nginx:latest

It's crucial to keep in mind that if the file or directory on the host machine does not exist, Docker will raise an error rather than creating the endpoint automatically.

Using Bind mounts and Volumes with Docker Compose

Bind-mount with Docker Compose

NodeJs application

services:
  frontend:
    image: node:lts
    volumes:
      - type: bind
        source: ./static
        target: /opt/app/static
volumes:
  myapp:

Volumes with Docker Compose

NodeJs application

services:
  frontend:
    image: node:lts
    volumes:
      - myapp:/home/node/app
volumes:
  myapp:

Postgres database

version: '3.8'
services:
  postgres:
    image: postgres:14-alpine
    container_name: "db-postgres"
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    volumes:
      - database_postgres:/var/lib/postgresql/data
 
volumes:
  database_postgres:

When using an externally created volume from another compose file or running: docker volume create.

services:
  frontend:
    image: node:lts
    volumes:
      - myapp:/home/node/app
volumes:
  myapp:
    external: true

Recap

To enhance container functionality, it's best to avoid relying on the default writable layer. Instead, consider utilizing Bind mounts and/or Volumes for improved data management within containers.

Docker Volumes are the recommended approach, especially when the data doesn't require access by other applications running outside of Docker on the host machine. Other advantages include:

• Managing them via the CLI (docker volume create)
• Safely shared across multiple containers
• Ability to backup, restore, or migrate data

If the situation demands it, utilize Bind mounts to facilitate data sharing between the Docker host and multiple containers, such as sharing source code, build artifacts, or configuration files located on the host machine.

Additional Resources

• Docker Storage Documentation