Docker compose windows 10 mount volume

It seems you are using an absolute path located inside C:Users dir, that didn’t work for me either, and if you are using Docker-Toolbox see below.

Overview

Forwarding the ./ relative path in volumes section will automatically get resolved by docker-compose to the directory containing docker-compose.yml file (for example, if your project is in %UserProfile%/my-project then ./:/var/www/html gets /c/Users/my-name/my-project:/var/www/html).

The problem is that currently (using DockerToolbox-19.03.1) only the /c/Users directory gets shared with the Virtual-Machine (toolbox puts docker itself in the VM, which means it has no access to your file system, except mounted shared-directories).

Conclusion

So, basically placing your project there (C:UsersYOUR_USER_NAME) should make ./ work.
But not even that worked for me, and we ended up with below _prepare.sh script:

#!/bin/bash

VBoxManage='/c/Program Files/Oracle/VirtualBox/VBoxManage'

# Defines variables for later use.
ROOT=$(dirname $0)
ROOT=$(cd "$ROOT"; pwd)
MACHINE=default
PROJECT_KEY=shared-${ROOT##*/}

# Prepares machine (without calling "docker-machine stop" command).
#
if [ $(docker-machine status $MACHINE 2> /dev/null) = 'Running' ]; then
    echo Unmounting volume: $ROOT
    eval $(docker-machine env $MACHINE)
    docker-compose down
    docker-machine ssh $MACHINE <<< '
        sudo umount "'$ROOT'";
    '
    "$VBoxManage" sharedfolder remove $MACHINE --name "$PROJECT_KEY" -transient > /dev/null 2>&1
else
    docker-machine start $MACHINE
    eval $(docker-machine env $MACHINE)
fi

set -euxo pipefail
"$VBoxManage" sharedfolder add $MACHINE --name "$PROJECT_KEY" --hostpath "$ROOT" -automount -transient


docker-machine ssh $MACHINE <<< '
    echo Mounting volume: '$ROOT';
    sudo mkdir -p "'$ROOT'";
    sudo mount -t vboxsf -o uid=1000,gid=50 "'$PROJECT_KEY'" "'$ROOT'";
'

docker-compose up -d
docker-machine ssh $MACHINE
bash

Usage:

• Place a copy of it beside each project’s docker-compose.yml file.
• Run it each time the system is turned on (simply double-click it or its shortcut).
• Done! relative paths should now work even if your project is in another drive (far away and outside of C:Users dir).

Note:

• With a little edit, it should work without docker-compose being required.
• Consider running docker system prune to free disk-space (or simply add docker system prune --force to the above script, on a new line right after mount command).

Docker is a common containerization solution that offers a user-friendly interface. It allows you to deploy your application as a lightweight process set rather than a complete virtual machine.

Docker images are like a snapshot of a container’s file system and contain both your application and its dependencies. When you run it, you recreate the container’s state. You don’t have to be concerned about setting up your environment because running an image recreates everything for you and is isolated from your operating system and other running containers.

The Docker interface is simple and users can easily create and implement applications into their containers or carry out version management, copy, share, and modify, just like managing ordinary code.

However, containers often need to use data beyond their container or share data between containers. While it may be tempting to rely on the host file system, a better solution is to work with persistent data in a container, namely Docker volumes.

A Docker volume is an independent file system entirely managed by Docker and exists as a normal file or directory on the host, where data is persisted.

In this guide, you’ll learn how volumes work with Docker, what they do, and what the best practices are for keeping them secure and effective.

What Are Docker Volumes

The purpose of using Docker volumes is to persist data outside the container so it can be backed up or shared.

Docker volumes are dependent on Docker’s file system and are the preferred method of persisting data for Docker containers and services. When a container is started, Docker loads the read-only image layer, adds a read-write layer on top of the image stack, and mounts volumes onto the container filesystem.

Why Docker Volumes?

If you are using Docker for development, you must be familiar with the -v or --volume flag that lets you mount your local files into the container. For instance, you can mount your local ./target onto the /usr/share/nginx/html directory container or an nginx container to visualize your html files.

echo "<h1>Hello from Host</h1>" > ./target/index.html
docker run -it --rm --name nginx -p 8080:80 -v "$(pwd)"/target:/usr/share/nginx/html nginx

Navigate to http://localhost:8080/ and you should see “Hello from Host”.

This is called a bind mount and is commonly used by developers. But, if you are using Docker Desktop on Windows or MacOS bind, mounts have significant performance issues. As a result, using volumes may be the best alternative for holding state between container runs.

Unlike bind mount, where you can mount any directory from your host, volumes are stored in a single location (most likely /var/lib/docker/volumes/ on unix systems) and greatly facilitates managing data (backup, restore, and migration). Docker volumes can safely be shared between several running containers.

You can also save data to a remote server or in cloud Docker volumes with alternative volume drivers like sshfs.

In addition, Docker enables you to manage volume with the command line docker volume, making their management simple.

Creating and Managing Docker Volumes

In this section, you’ll learn how to create a Docker volume implicitly and explicitly and then declare it from a Docker file. Then you’ll learn how to view a data volume, mount it to a container, and configure it using docker-compose.

Create a Docker Volume Implicitly

The easiest way to create and use a volume is with docker run and the -v or --volume flag. This flag takes three arguments separated by ::

-v <source>:<destination>:<options>

If the “source” is a path that was used in the previous example, Docker will use a mount bind. If the “source” is a name, then Docker tries to find this volume or creates one if one cannot be found. Below, the previous example has been updated to use a volume instead of a mount bind:

docker run -it --rm --name nginx -p 8080:80 -v demo-earthly:/usr/share/nginx/html nginx

You can check to make sure the container was properly created with docker volume ls which lists all existing volumes.

Note that the volume in question is not empty. If a volume is completely empty, the container’s content is copied to the volume.

You can check the status of your volumes on Linux. This gives you a chance to see where volumes are stored:

ls /var/lib/docker/volumes/target/_data/demo-earthly

On Mac and Windows it’s a bit more tricky. In order to keep things simple, you can mount the volume on an ubuntu container and use ls to see the content of your volume:

docker run -it --rm -v demo-earthly:/opt/demo-earthly ubuntu ls /opt/demo-earthly

Create a Docker Volume Explicitly

Alternatively you can use the docker volume create command to explicitly create a data volume. This command gives you the option to choose and configure the volume driver. The implicit creation of volumes always uses the local driver with default settings.

docker volume create --name demo-earthly

Declare a Docker Volume from Dockerfile

Volumes can be declared in your Dockerfile using the VOLUME statement. This statement declares that a specific path of the container must be mounted to a Docker volume. When you run the container, Docker will create an anonymous volume (volume with a unique id as the name) and mount it to the specified path.

FROM nginx:latest

RUN echo "<h1>Hello from Volume</h1>" > /usr/share/nginx/html/index.html
VOLUME /usr/share/nginx/html

Lets build and run your new image:

docker build -t demo-earthly .
docker run -p 8080:80  demo-earthly

You can now validate that nginx serves your message at http://localhost:8080/.

More importantly, an anonymous Docker volume has been created, and every time you start a new container, another volume is created with the content of /usr/share/nginx/html.

From the above example, ​​a volume directory data with the text file test containing “Hello from Volume” is created.

View a Data Volume

To manage your data, sometimes you need to list data volumes from the command line as a point of reference, which is faster than repeatedly checking the configuration files. You can use the docker volume ls command to view a list of data volumes.

Use the docker volume inspect command to view the data volume details.

Mount a Volume to a Container

As you have seen through the various examples -v and --volume are the most common way to mount a volume to a container using the syntax:

-v <name>:<destination>:<options>

One notable option is ro which means that the volume will be mounted as read-only:

docker run -it -v demo-volume:/data:ro ubuntu

Try to write into the folder/data to validate that the volume is in read-only mode:

An alternative to -v is to add the —mount option to the docker run command. --mount is the more verbose counterpart of -v.

To launch a container and mount a data volume to it, follow this syntax:

docker run --mount source=[volume_name],destination=[path_in_container] [docker_image]

Replace [path in container] with the path to attach the Docker volume [volume_name] in the container.

For example, run the following command to start an Ubuntu container and mount the data volume to it.

docker run -it --name=example --mount source=demo-volume,destination=/data ubuntu

Remember if the volume doesn’t exist Docker will create it for you.

List the contents of the container to see if the volume is mounted successfully. You should find the Docker volume name defined in the above data syntax.

Configure a Volume Using docker-compose

Although there are many ways to create a volume, it’s more convenient to use the docker-compose command to easily share data between multiple containers.

The use of the volume property in compose files is very similar to -v and --volume. That being said, to perform a bind mount (mount a directory from your local machine), you can use a relative path unlike -v with the command docker run that requires an absolute path.

version: "3.2"
services:
  web:
    image: nginx:latest
    ports:
      - 8080:80
    volumes:
      - ./target:/usr/share/nginx/html

The containers and hosts in the above configuration use volumes in the services definition (web) to mount ./target from the host to /usr/share/nginx/html of the container. As with the first example, if you navigate to http://localhost:8080/ you should read “Hello from Host”.

With docker-compose, volumes must be declared at the same level as services. Then you can refer to them by their name.

version: "3.2"
services:
  web:
    image: nginx:latest
    ports:
      - 8080:80
    volumes:
      - html_files:/usr/share/nginx/html
  web1:
    image: nginx:latest
    ports:
      - 8081:80
    volumes:
      - html_files:/usr/share/nginx/html
 
volumes:
  html_files:

In this example, you declared a volume named html_files and used it in both web and web1 service. Multiple containers can mount the same volume.

Running docker-compose up will create a volume named <project_name>_html_files if it doesn’t already exist . Then run docker volume ls to list the two volumes created, starting with the project name.

You can also manage container outside of you docker-compose file, but you still need to declare them under volumes and set the property external: true.

version: "3.2"
services:
  web:
    image: nginx:latest
    ports:
      - 8080:80
    volumes:
      - html_files:/usr/share/nginx/html
 
volumes:
  html_files:
    external: true

If you don’t have html_files, you can use docker volume create html_files to create it. When you add external, Docker will find out if the volume exists; but if it doesn’t, an error will be reported.

Copy Files Between Containers from a Shared Volume

Let’s look at how Docker volumes enable file sharing across containers.

In this example, use the volume and container we previously defined and execute the following commands:

docker create volume demo-earthly

docker run -it --name=another-example --mount source=demo-volume,destination=/data ubuntu

Navigate to the data volume directory and create a file using the command touch demo.txt. Exit the container, then launch a new container another-example-two with the same data volume:

docker run -it --name=another-example-two --mount source=demo-volume,destination=/data ubuntu

The demo.txt file you created in the preceding container should list another-example in the output.

Docker Volume Best Practices

Now that you’ve learned how to implement Docker volumes, it’s important to keep in mind a few best practices:

• Always mount volumes as read-only if you only need to read from them.
• Always set the permissions and ownership on a volume.
• Always use environment variables for the host path or volume name in a production environment.

Conclusion

Often, you want your containers to use or persist data beyond the scope of the container’s lifetime. You can use volumes to solve this problem by working with Docker to mount, create, and share volumes between containers.

In this guide, you looked at how volumes work with Docker, what they do, and where volumes are the preferred solution.

Docker plays an important role in the DevOps ecosystem, so if you want to improve your continuous integration process, consider Earthly. Earthly is a build automation tool that allows you to build anything via containers, making your builds self-contained, repeatable, portable, and parallel.

Volumes store the data generated by Docker. Using Docker Volume, we can achieve data persistence within Docker Containers. We can share directories between container and container versions using Docker Volumes.  Also, we can upgrade containers, restart machines or share data without losing it. So let us discuss more of the Data persistence and Docker Volumes in this article. We will cover the following topics as part of this tutorial.

• How to manage data in Docker?
  • Volume vs. Bind Mount in Docker
  • How to use Mounts in Docker?
  • How to use data volumes?
• The -v or -mount flag?
• Create and manage Docker data volumes.
• How to use data volumes with docker-compose?
• What are read-only data volumes?
• Start a container/service with Docker data volume
• How to populate a Docker data volume using a container?
• What is a Volume driver?
  • Initial set-up — docker plugin install command
  • Create a data volume using a volume driver — docker volume create command
  • Start a container that creates a volume using a volume driver
• Backup, restore, migrate, or remove data volumes in Docker
  • Backup a container
  • Restore container from backup
  • Remove docker volume

We know that Docker has a layered storage implementation that aids in portability, efficiency, and faster performance. This storage implementation is also optimal for retrieving, storing, and transferring images across various environments. When we delete a Docker container, all the data associated or written to the container is deleted with it. So there is a need to persist the container data somehow even when the container gets deleted so that we need not worry about data and persist this data after the container ceases to exist.

Hence there is a need to persist data in containers. One of the best-recommended practices is the isolation of data from the container to retain the benefits of containerization. Here we keep data management distinctly separate from container lifecycle. We can implement multiple strategies to persist data or add persistence to containers that we will discuss next. These strategies are as shown in the diagram below.

As shown above, Docker provides two options for data persistence so that files are persisted even after the container stops.

• Volumes
• Mounts

Let’s understand the difference between these two:

Volume vs. Bind Mount in Docker

1. Volumes: Volumes are directories or files that are outside the Union file system(the combination of read-only layers with a read-write layer on top of the container). Volumes exist/ store as normal files and directories on the host filesystem.  Hence to persist and share data between containers, Docker uses Volumes. Volumes are the best option to persist data in Docker containers. Docker manages volumes and is stored in a part of the host filesystem (/var/lib/docker/volumes/ on Linux).

Note: Processes not part of Docker (non-docker processes) should not modify this portion of the filesystem.

1. Mounts: In Docker, we can use the following mounts.

• Bind mounts:  A file or folder stored anywhere on the container host filesystem and mounted into a running container is called a bind mount. They can be anything from essential system files to directories.  Interestingly, Docker container or Non-Docker processes present on the Docker host can modify these mounts at any time.
• tmpfs mounts are used mainly by Docker running on Linux systems. Their storage is in the host system’s memory only. Additionally, we never write the tmpfs mounts to the host system’s filesystem. Contrary to volumes and bind mounts, the «tmpfs» mount is temporary and only persisted in the host memory. When the container stops, the «tmpfs» mount removes, and files written there won’t persist.

Note: If the Docker is running on Windows, we can use the named pipe.

The Docker volumes are widely used and useful tools for data persistence when working with Docker containers. Compilation of additional writable layers adds to the image size, and hence volumes are the better alternative. Secondly, the volumes are independent of the container life cycle and stored on the host. Because of this, we can back up data and share file systems between one or more containers easily.

For now, just keep in mind these methods, as the details will be shared later in the post.

How to use Mounts in Docker?

When we use bind mounts, we can control the exact mount point, a directory (typically an empty one) in the currently accessible filesystem on which additional filesystem mounts (i.e., logically attached) we want the host system. Apart from using bind mounts to persist data, it often provides additional data to containers. Using bind mounts to mount source code into the container lets us see code changes right away and respond.

The question now arises as to which mount should we use and when?

Although we should generally use volumes as far as possible, bind mounts are appropriate in the following use cases:

• When we need to share configuration files to containers from the host machine. For instance, to provide DNS resolution to containers, Docker mounts «/etc/resolv.conf» into each container from the host.
• When we want to share the build artifacts or source code between the development environment and container on the host machine. For example, we may mount, say, the Maven ‘target/’ directory into a container. Thus each time we build the Maven project on the host, the container will access the updated artifacts.
• We can also use bind mounts when the directory or file structure of the Docker host is consistent with the bind mounts required by containers.

So discussed above are some of the cases or situations where we find bind mounts useful.

On the other hand, we can use tmpfs mounts when we do not want to persist data on the host machine and the container. It is mainly for security reasons and protects the container performance, especially when the application has to write large non-persistent data volumes.

As evident from the above discussion, we use tmpfs mounts seldomly. Nonetheless, we should keep in mind a few tips when working with bind mounts and volumes.

• When we mount an empty volume into a directory inside the container, the files and/or directories inside the container propagate into the volume.
• When we start the container and specify a volume that does not exist, Docker creates an empty volume.
• Similarly, when we mount non-empty volumes or bind mounts into a directory inside the container which already has some files and/or directories, then these files and/or directories are obscured by the mount and are not accessible while the volume or bind mount is mounted.

In the remainder of this article, we will discuss volumes in detail.

How to use data volumes?

Volumes are popular and preferred mechanisms for data persistence in Docker containers. As against bind mounts dependent on the OS and directory structure of the host machine, volumes are managed entirely by the Docker. Some of the advantages of volumes over bind mounts are as follows:

• We manage volumes using Docker API commands.
• Volumes are supported both by Linux and Windows.
• It is easier to back up or migrate volumes as compared to bind mounts.
• We can quickly and safely share volumes among multiple containers.
• Using volume drivers, we can also store volumes on remote hosts or cloud providers. We can also encrypt the contents of volumes or add any other functionality.
• The container can pre-populate the new volumes.
• Volumes on Docker Desktop perform more efficiently than bind mounts from Windows or Mac hosts.

Volumes do not increase the size of the containers that are using them, and also, contents of the volume exist outside the lifecycle of that container. Hence volumes are a popular choice for persisting data. The following diagram shows volumes and mounts in action.

So if the container has to store non-persistent data that is temporary, we should ideally consider using tmpfs mount. For other things like storing data permanently or writing it into the container’s writable layer, we go for volumes or bind mounts.

The -v or -mount flag?

As far as volumes are concerned, they use private bind propagation, and we cannot configure bind propagation for volumes.

As to the options —mount or —v, —mount is more verbose and explicit. The main difference between —v and —mount is that the —v syntax combines all options in one field, whereas the —mount syntax separates them.

For specifying volume driver options, we should use —mount.

1. -v or —volume: This option consists of three fields separated by a colon (:). We must ensure that the fields are in the correct order. Field descriptions are as below.

• For anonymous volumes, the first field omits. In the case of named volumes, the first field contains the name of the volume. The name is unique for a given host machine.
• The second field contains the path of the location of the file or directory mounted in the container.
• The third field is optional. It contains a comma-separated list of options, such as ro.

1. —mount: This option contains multiple key-value pairs, separated by commas (,) with each consisting of (<key>=<value>) tuple. Note that the syntax for ‘—mount‘ is more verbose than ‘—v‘ or ‘—volume‘.

• The type (type) of the mount: This can be bind, volume, or tmpfs.
• The source of the mount(source/src): For named volumes, this field contains the volume’s name. We omit this field for anonymous volumes.
• The destination(destination/dst/target): This field has the value of the path where the directory or file is mounted.
• The readonly option: When present, this field makes the bind mount to mount as read-only.
• The volume-opt option: This field consists of key-value pairs with option name and value. It can be specified more than once.

Now let us tabularise the main differences between —v and —mount options.

Note: Contrary to bind mounts, all the options for volumes are present for —mount as well -v flags. When we use volumes with services, only —mount is supported.

In the next section, let us go through Docker commands related to volumes.

Create and manage Docker data volume

Volumes can be created and managed outside the scope of any container. To create a docker volume, use the ‘docker volume create‘ command on the console. The command to is given below.

$ docker volume create my-vol

We can list the existing volumes using the following command.

$ docker volume ls

Given a volume, we can retrieve its details by using the «docker volume inspect command«.

$ docker volume inspect volume_name

The following screenshot shows the working of the above commands.

As shown in the above screenshot, we have created a new volume, «my_docker_volume» with the docker volume create command:

docker volume create my_docker_volume

The next command, «docker volume ls«, lists the volume just created. Next, we specify the command:

docker volume inspect my_docker_volume

This command gives the details of the volume, as shown in the above screenshot.

Similarly, we can use the following command to remove the volume.

$ docker volume rm Volume_name

Hence to remove the volume created above, we can specify the command,

docker volume mu_docker_volume

The result of the command is as shown below.

To verify that we indeed removed the volume, we can give the command «docker volume ls«.

How to use data volume with docker-compose?

We can also create a volume with Docker compose service or also specify existing volumes. For example, the following screenshot shows a ‘docker-compose‘ file that creates a docker-compose service with a volume.

The command is as follows:

docker-compose up

Output generated:

As a result of the above command, a volume with the name ‘myvolume‘ gets created. Subsequent invocations of ‘docker-compose up‘ will reuse this volume.

Alternatively, we can also create a volume outside of Docker Compose and then reference it inside the ‘docker-compose.yaml‘ file, as shown in an example below.

version: "3.3" 
services: 
frontend: 
image: node:lts 
volumes: 
- myvolume:/home/node/app 
volumes: 
myvolume: 
external: true

In the above docker-compose file, we have used the volume with the name ‘myvolume‘. Note that a flag ‘external‘ is specified and is set to true that indicates the volume already created outside the docker-compose.

What are read-only data volumes?

In most cases, the container only needs access to the data for reading purposes. Also, some folders may change a lot, and it might be difficult and costly to create images for each execution. In this situation, we go for read-only volumes.

To specify the volume as read-only, we append ‘ro‘ to the -v switch as shown below:

docker run -v volume-name:/path/in/container:ro my/image

We can also use the «mount» switch with the ‘readonly‘ option, as shown below.

$ docker run --mount source=volume-name,destination=/path/in/container,readonly my/image

We use read-only volumes in the case of dynamic data used in the application.

Start a container/service with Docker data volume

When we start a container with the non-existing volume, Docker creates a new volume with the specified name. The following example shows this.

Here we give the following command.

docker run -d --name volumetest -v my_docker_volume:/app ubuntu:latest

As shown in the above command, a new container, «volumetest«, is started with the specified volume name as «my_docker_volume«. This volume is non-existing. On execution of the above command, we get the ID of the container volumetest. To verify that we actually created  the volume, we can give the inspect container command as follows:

docker inspect volumetest

When this command executes, we can directly jump to the «Mounts» section of the inspect output, as shown in the below screenshot.

As seen in the above screenshot, we have the details of the volume created. volume’ and the read-write option is true for this mount. We can then stop the container and remove the volume with the following sequence of commands.

$ docker container stop volumetest

$ docker container rm volumetest

$ docker volume rm my_docker_volume

When a service starts and we define a volume, each service container will use its local volume. So if we use a local volume driver, no container will share the data. But there are exceptions wherein some volume drivers support shared storage. For example, Docker for AWS and Azure uses the Cloutstor plugin to support shared persistent storage.

Starting the service with the local volume is similar to that of starting a container. The difference is that in the ‘docker service create‘ command, the ‘-v‘ or ‘—volume‘ flag is not supported. Instead, we must use the ‘—mount‘ flag for mounting the volume.

How to populate a Docker data volume using a container?

When we start a container that creates a new volume, and if the container has files and directories inside it, then the contents of this directory are copied into the volume. Thus, the container that mounts this volume and other containers that use this volume will have access to pre-populated contents.

As an example, we give the following command to start the ubuntu container and populate the new volume «ubuntu_vol» with the contents of the «/usr/share/ubuntu/html» directory.

docker run -d --name=mounttest --mount source=ubuntu_vol,
              destination=/usr/share/ubuntu/html ubuntu:latest

After this, we can clean up the containers and volumes by executing the following sequence of commands.

$ docker container stop mounttest

$ docker container rm mounttest

$ docker volume rm ubuntu_vol

Apart from this command, we can also prepopulate the volume with the command used in the previous section to start the container.

What is a Volume driver?

We can use volume drivers when we need to store volumes on remote hosts or cloud providers to encrypt the contents of volumes or add more functionality. For example, when we use the «docker volume create» command, or we start a container with a non-existing volume, we can specify the volume driver. One of the examples of the volume driver is «Vieux/sshfs«. In this section, we will just touch base with the basics of volume drivers.

Initial set-up — docker plugin install command

The first step in using the volume driver is to install the appropriate plugin. For example, we can install the plugin vieux/sshfs with the following command.

$ docker plugin install --grant-all-permissions vieux/sshfs

It will install the volume driver plugin on the docker host.

Create a data volume using a volume driver — docker volume create command

The next step is to use the just installed plugin to create a volume.

$ docker volume create --driver vieux/sshfs 
  -o [email protected]:/home/test 
  -o password=testpassword 
  sshvolume

We have specified an SSH password in the above command, but we can omit it if two hosts configure with shared keys. The flag ‘-o‘ specifies zero or more configurable options (like username and password in the above command).

Start a container that creates a volume using a volume driver

After installing the plugin, we can also start a container by specifying the volume driver’s non-existing volume. Docker will create a new volume while starting the container. The following example shows creating an Nginx container with the «vieux/sshfs» volume driver.

$ docker run -d --name sshfs-container --volume-driver vieux/sshfs --mount src=sshvolume,
         target=/app,[email protected]:/home/test,
         volume-opt=password=testpassword nginx:latest

Again we can omit the password if there are shared keys, and there can be zero or more configurable options specified with ‘volume-opt‘. Remember to use ‘—mount‘ flag to mount the volume when the volume driver needs to pass options.

Backup, restore, migrate, or remove a data volume in Docker

We can use volumes for migrations, backups, and restores. Let us discuss these with volumes.

Backup a container

We follow the following steps to back up a container.

1. Create a new container
2. Launch a new container and mount the volume from the container created in step 1.
3. Mount localhost directory as /backup
4. Tar the contents of the volume to backup.tar file inside the /backup directory.

Thus we have a backup of the volume in /backup local directory. Let us take an example to illustrate these commands.

First, we create a new container with the following command:

docker run -v /dbdata --name dbcontnr ubuntu /bin/bash

Next, we can create a backup directory or backup the volume in the present directory itself.  Once this happens, we execute the following command to tar the volume contents.

docker run --rm --volumes-from dbcontnr -v $(pwd):/backup ubuntu tar 
cvf /backup/backup.tar /dbdata

The following screenshot shows the result of the execution of the above commands.

When we list the contents of the current directory, we can see the backup.tar file of dbdata volume created on the host machine.

Restore container from backup

Now that we have created a backup, how do we restore it to the container? We can either restore the backup to the exact container or another container. First, let us restore the backup we created in the previous section to a new container. For this first, we create a new container as follows.

docker run -v /dbdata --name dbcontainer2 ubuntu /bin/bash

Now we restore the backup (backup.tar) to this container as follows:

docker run --rm --volumes-from dbcontainer2 -v $(pwd):/backup ubuntu bash -c 
     "cd /dbdata && tar xvf /backup/backup.tar --strip 1"

Once the command is successful, the container ‘dbcontainer2’s‘ data volume will have the contents of untar file into it.

We can use these techniques of backup, migration, and restore to automate the entire process.

Remove docker volume

The data volume of a Docker container persists even after the container deletes. A Docker data volume persists after a container deletes. We have two types of data volumes:

• Named volumes: Named volumes have a specific source that is outside the container, for example awesome : /hello.
• Anonymous volumes: These volumes do not have a specific source when the container deletes. We should instruct the Docker Engine Daemon in case we need to remove them.

To remove anonymous volumes automatically, we use the «— rm» option. For example, if we have an anonymous /bkup volume. Then, when the container deletes, the Docker Engine will automatically remove the /bkup volume with the following command.

$ docker run --rm -v /bkup -v awesome:/hello busybox top

The above command creates an anonymous /bkup volume. So when we remove the container, the Docker Engine removes the /bkup volume but not the awesome volume.

To remove all the unused volumes, we can use the following command:

$ docker volume prune

It removes all the volumes that are unused and frees up space.

Key TakeAways

• There are mounts that we can also use for persisting data apart from volumes.
• When we create a container and specify a non-existing container, the Docker creates the volume for us.
• We can also create a volume and then use it with a container using the -v flag.
• Docker-compose allows us to use volumes that are either existing or new.
• Using volumes, it is easier to backup, migrate and restore data and even automate the entire process.

Данная публикация является разбором особенностей контейнерной виртуализации Docker под системой Windows.

Она не претендует на роль исчерпывающей и по мере необходимости будет обновляться и дополняться.

За практическим руководством с нуля советую обратиться к этой публикации.

Содержание

• Предварительные настройки
• Выбор между Docker Toolbox on Windows или Docker for Windows
• Windows контейнеры и Linux контейнеры
• Особенности монтирования папок
• Монтирование с хост-машины или volume
• Особенности разметки диска GPT и MBR
• Docker Toobox to Windows
• Docker Swarm
• Проблемы с кодировкой
• Полезные ссылки
• Заключение

Предварительные настройки

Контейнерная виртуализация или виртуализация на уровне операционной системы Docker нативно работает только на дистрибутивах Linux и FreeBSD (экспериментально).
На Windows вам понадобится гостевая Linux система либо специальная минималистичная виртуальная машина с ядром Linux от разработчиков Docker, которая и ставится из коробки.
Само собой разумеется, что вы включили виртуализацию у себя в BIOS/UEFI
Пункт настройки может называться по-разному: VT-x, VT-d, Intel VT, AMD-V, Virtualization Technology.

Еще одним минимальным системным требованием будет разрядность системы x64 и версия не ниже Windows 7 Pro.

Выбор между Docker Toolbox on Windows или Docker for Windows

Появление Docker Toolbox on Windows и Docker Toolbox on Mac было большим событием.

Сборка включается в себя сам docker, утилиту docker-compose, утилиту для работы с виртуальной машиной docker-machine и клиент Kitematic.

Используется виртуальная машина (по умолчанию на VirtualBox) с минималистичным Linux окружением.

Позже для новых операционных систем выпустили Docker for Windows и Docker for Mac, которая на текущий момент является актуальной версией и продолжает развиваться.

Выбор между версиями не сложный:
— Если у вас Windows 10 x64 Pro, Enterprise или Education то включаем службу Hyper-V и ставим Docker for Windows.

Заметьте, что после включения службы Hyper-V пропадет возможность запускать и создавать x64 виртуальные машины на VirtualBox.

— Если же у вас другая версия Windows(7 Pro, 8, 8.1, 10 Home) то ставим VirtualBox и Docker Toolbox on Windows.

Несмотря на то, что Docker Toolbox разработчиками признан устаревшим работа с ним слабо отличается от Docker for Windows.

Вместе с установкой Docker Toolbox будет создана виртуальная машина.
В самом VirtualBox можно будет добавить оперативной памяти и ядер процессора на ваше усмотрение.

Windows контейнеры и Linux контейнеры

Docker for Windows предоставляет возможность переключать контейнеризацию между Linux и Windows версией.

В режиме Windows контейнеризации вы можете запускать только Windows приложения.
Замечу, что на май 2018 года в официальном Docker Hub существует всего 13 образов для Windows.

После включения Windows контейнеризации не забудьте добавить внешнюю сеть.

В конфигурационном файле docker-compose.yml это выглядит так:

networks:
  default:
    external:
      name: nat

Особенности монтирования папок

На примонтированных volume-ах не кидаются события файловой системы, поэтому inotify-tools не работает.
Спасибо пользователю eee

Если вы разрабатываете свой проект и пользуетесь docker-compose вне домашней папки то вам нужно будет проделать некоторые манипуляции.

Используя Docker for Windows для монтирования нового диска у вашего локального пользователя обязательно должен стоять пароль, который будет использоваться для доступа к shared папки.

Особенность заключается в том, что монтируемые внутрь контейнера диск будет монтироваться как от удаленной машины //10.0.75.1/DISK_DRIVE по протоколу SMB.

Для Docker Toolbox диски монтируются в самом VirtualBox на вкладке «Общие папки»
Пример для диска «D»:

Права доступа к монтируемым файлам и папкам

Как бы вам не хотелось, но для всех примонтированных из хост-машины файлов и папок будут стоять права 755 (rwx r-x r-x) и поменять их вы не сможете.

Остро встает вопрос при монтировании внутрь файла закрытого SSH ключа, права на который должны быть только у владельца(например 600).

В данном случае либо генерируют ключ при создании образа, либо прокидывают сокет ssh-agent с хост-машины.

Монтирование с хост-машины или volume

Монтирование внутрь контейнера происходит с использованием сети и протокола SMB, следовательно, внутри контейнера диск «D:» будет примонтирован из источника //10.0.75.1/D
Использование volume внутри контейнера отображается как монтирование локального диска /dev/sda1, что влияет на скорость работы.

Простым тестом копирование файла на обычном HDD скорость работы получилась следующая:

Такая разница в скорости скорее всего связана с тем, что в volume данные сбрасываются на диск постепенно, задействуя кеш в ОЗУ.

Особенности разметки диска GPT и MBR

Данный пункт не является истинной так как опровергающей или подтверждающей информации в интернете найти не смог.

Если на хост-машине таблица разделов MBR, то контейнер с MySQL/MariaDB может упасть с ошибкой:

InnoDB: File ./ib_logfile101: ‘aio write’ returned OS error 122. Cannot continue operation

По умолчанию в базе данных включеён параметр innodb_use_native_aio, отвечающий за асинхронный ввод/вывод и его надо будет выключить.

Данная проблема также встречается на некоторых версиях MacOS.

Docker Toobox to Windows

Главное правило: начинать работу с запуска ярлыка на рабочем столе «Docker Quickstart Terminal», это решает 80% проблем.

— Бывает возникают проблемы с отсутствия переменных окружения, решается командой:

eval $(docker-machine env default)

— Если все же возникают проблемы из разряда «docker: error during connect», необходимо выполнить:

docker-machine env --shell cmd default 
@FOR /f "tokens=*" %i IN ('docker-machine env --shell cmd default') DO @%i

Название Docker Machine по умолчанию default.

Docker Swarm

Ни в Docker for Mac, ни в Docker for Windows — нет возможности использовать запущенные демоны в качестве клиентов кластера (swarm members).
Спасибо пользователю stychos

Проблемы с кодировкой

Используя Docker Toolbox(на Docker for Windows не удалось воспроизвести) нашлась проблема с тем, что русские комментарии в docker-compose.yml файле приводили к ошибке:

Traceback (most recent call last):
  File "docker-compose", line 6, in <module>
  File "composeclimain.py", line 71, in main
  File "composeclimain.py", line 124, in perform_command
  File "composeclicommand.py", line 41, in project_from_options
  File "composeclicommand.py", line 109, in get_project
  File "composeconfigconfig.py", line 283, in find
  File "composeconfigconfig.py", line 283, in <listcomp>
  File "composeconfigconfig.py", line 183, in from_filename
  File "composeconfigconfig.py", line 1434, in load_yaml
  File "site-packagesyaml__init__.py", line 94, in safe_load
  File "site-packagesyaml__init__.py", line 70, in load
  File "site-packagesyamlloader.py", line 24, in __init__
  File "site-packagesyamlreader.py", line 85, in __init__
  File "site-packagesyamlreader.py", line 124, in determine_encoding
  File "site-packagesyamlreader.py", line 178, in update_raw
  File "c:projectscomposevenvlibencodingscp1251.py", line 23, in decode
UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 1702: character maps to <undefined>
[4176] Failed to execute script docker-compose

Полезные ссылки

Docker Toolbox on Windows
Docker for Windows
Практическое руководство по Docker

Заключение

Особенности работы с Docker контейнеризацией на системе Windows не отличается от работы на Linux за исключение разобранных выше.

В статье я умышленно не упомянул заметно низкую скорость работы контейнеров и overhead используя систему Windows как само собой разумеющееся.

Буду рад услышать ваши отзывы. Не стесняйтесь предлагать улучшения или указывать на мои ошибки.

Какой версией Docker вы пользуетесь?

Проголосовали 266 пользователей.

Воздержались 189 пользователей.