Lets learn "About docker directory structure"

-


 In this tutorial, we'll try to answer the following questions regarding the docker.

  1. Where docker images are stored in the filesystem?
  2. Where Docker containers are stored in the filesystem?
  3. How docker layers are stored and reused for different containers?
So here we'll try to answer all these questions. Before proceeding further I would recommend learning about docker image/container layers work in this tutorial, if you are not familiar already.

To get the basic information about the docker info run the below command.

$ docker info
   ...
   Storage Driver: overlay2
   Docker Root Dir: /var/lib/docker
   ...


The above command gives the docker's storage driver and docker root directory. 

Storage location for Docker images and containers

Storage location for docker images & containers is different in different operating systems. 

Ubuntu: /var/lib/docker
Debian: /var/lib/docker
Fedora: /var/lib/docker
MacOS: /Library/Containers/com.docker.docker/Data/vms/0
Windows: C:\ProgramData\DockerDesktop

There is a very interesting point for these locations in the case of Windows/macOS. On these operating systems docker info command will return /var/lib/docker as the Docker root directory even the actual root directories are different.
The reason behind that is in macOS and windows, docker runs Linux containers in a virtual environment.

In macOS, docker is not natively compatible, so Hyperkit ( https://github.com/moby/hyperkit ) is used to run run a virtual image. Its virtual image data is located in /Library/Containers/com.docker.docker/Data/vms/0 and within the virtual image docker path is /var/lib/docker that is what docker info reports.

The storage directory structure for Docker images and containers

Basically, let's see what is contained in the Docker root directory.

$ ls -ltrh /var/lib/docker
total 192K
drwx--x--x   3 root root 4.0K Aug 31 18:38 containerd
drwx------   4 root root 4.0K Aug 31 18:38 plugins
drwx------   3 root root 4.0K Aug 31 18:38 image
drwx------   2 root root 4.0K Aug 31 18:38 trust
drwxr-x---   3 root root 4.0K Aug 31 18:38 network
drwx------   2 root root 4.0K Aug 31 18:38 swarm
drwx------   2 root root 4.0K Aug 31 18:38 builder
drwx------   2 root root 4.0K Oct  1 19:31 tmp
drwx------   2 root root 4.0K Oct  1 19:31 runtimes
drwx------   5 root root 4.0K Oct  1 19:34 volumes
drwx------ 752 root root 104K Oct  3 01:19 overlay2
drwx------ 302 root root  40K Oct  3 01:19 containers

$ docker image pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
d72e567cc804: Pull complete
0f3630e5ff08: Pull complete
b6a83d81d1f4: Pull complete
Digest: sha256:bc2f7250f69267c9c6b66d7b6a81a54d3878bb85f1ebb5f951c896d13e6ba537
Status: Downloaded newer image for ubuntu:latest

$ docker image inspect ubuntu
[
    {
        "Id": "sha256:9140108b62dc87d9b278bb0d4fd6a3e44c2959646eb966b86531306faa81b09b",
        "RepoTags": [
            "ubuntu:latest"
        ],
        "RepoDigests": [
            "ubuntu@sha256:bc2f7250f69267c9c6b66d7b6a81a54d3878bb85f1ebb5f951c896d13e6ba537"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2020-09-25T22:34:30.295807036Z",
        "Container": "1046a5d685aef5c37d1829040ca8083b94e4c069ca4963f4b16a6ade2e077b06",
        "ContainerConfig": {
            "Hostname": "1046a5d685ae",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"/bin/bash\"]"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:4ff2090064e7e38688bce713d50f3202d227b3c89fecea1434271c912ccd47e0",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "DockerVersion": "18.09.7",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/bash"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:4ff2090064e7e38688bce713d50f3202d227b3c89fecea1434271c912ccd47e0",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 72875723,
        "VirtualSize": 72875723,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/9ebf430798b5bee2d05e18820c713efc75c3b6a5db9cde270dd065a7a85cc009/diff:
/var/lib/docker/overlay2/24b4b2c0d4479569897046447332ebd63d7c72b15af5a8e7b73fc494fbc297bd/diff",
                "MergedDir": "/var/lib/docker/overlay2/3ee2e31d2b5eeafd9cbe6c99bcf1d2f7121c272beebf7a088fede8e564fcd368/merged",
                "UpperDir": "/var/lib/docker/overlay2/3ee2e31d2b5eeafd9cbe6c99bcf1d2f7121c272beebf7a088fede8e564fcd368/diff",
                "WorkDir": "/var/lib/docker/overlay2/3ee2e31d2b5eeafd9cbe6c99bcf1d2f7121c272beebf7a088fede8e564fcd368/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:d42a4fdf4b2ae8662ff2ca1b695eae571c652a62973c1beb81a296a4f4263d92",
                "sha256:90ac32a0d9ab11e7745283f3051e990054616d631812ac63e324c1a36d2677f5",
                "sha256:782f5f011ddaf2a0bfd38cc2ccabd634095d6e35c8034302d788423f486bb177"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]


Let's discuss the Data part in the above output. It has

LowerDir: This directory contains the read-only layers of an image.  From above diagram we have two entries in LowerDir section:
    1. /var/lib/docker/overlay2/9ebf430798b5bee2d05e18820c713efc75c3b6a5db9cde270dd065a7a85cc009/
    2. /var/lib/docker/overlay2/24b4b2c0d4479569897046447332ebd63d7c72b15af5a8e7b73fc494fbc297bd/
Last in this list is the base image lower and top in the list is top image layer. lets see the content of each entry in the lower Dir

$ ls /var/lib/docker/overlay2/24b4b2c0d4479569897046447332ebd63d7c72b15af5a8e7b73fc494fbc297bd/
diff  link

$ ls /var/lib/docker/overlay2/24b4b2c0d4479569897046447332ebd63d7c72b15af5a8e7b73fc494fbc297bd/diff
bin  boot  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

$ cat /var/lib/docker/overlay2/24b4b2c0d4479569897046447332ebd63d7c72b15af5a8e7b73fc494fbc297bd/link
HCB7TN3IBHW6BJHN2B65F2N27E

$ ls -l /var/lib/docker/overlay2/l/ | grep HCB7TN3IBHW6BJHN2B65F2N27E
lrwxrwxrwx 1 root root 72 Oct  4 09:57 HCB7TN3IBHW6BJHN2B65F2N27E -> ../24b4b2c0d4479569897046447332ebd63d7c72b15af5a8e7b73fc494fbc297bd/diff

../diff contains the content of the layer. ../link file contains content of symlink to current layer.

$ ls /var/lib/docker/overlay2/9ebf430798b5bee2d05e18820c713efc75c3b6a5db9cde270dd065a7a85cc009/
diff  link  lower  work

$ ls /var/lib/docker/overlay2/9ebf430798b5bee2d05e18820c713efc75c3b6a5db9cde270dd065a7a85cc009/diff
etc  usr  var

$ ls /var/lib/docker/overlay2/9ebf430798b5bee2d05e18820c713efc75c3b6a5db9cde270dd065a7a85cc009/work

$ cat /var/lib/docker/overlay2/9ebf430798b5bee2d05e18820c713efc75c3b6a5db9cde270dd065a7a85cc009/link
TFIREK5IHQQ66COROUJXYZSSBT

$ cat /var/lib/docker/overlay2/9ebf430798b5bee2d05e18820c713efc75c3b6a5db9cde270dd065a7a85cc009/lower
l/HCB7TN3IBHW6BJHN2B65F2N27E

$ ls -l /var/lib/docker/overlay2/l/ | grep TFIREK5IHQQ66COROUJXYZSSBT
lrwxrwxrwx 1 root root 72 Oct  4 09:57 TFIREK5IHQQ66COROUJXYZSSBT -> ../9ebf430798b5bee2d05e18820c713efc75c3b6a5db9cde270dd065a7a85cc009/diff

Again ../diff contains the content of layer. link contains symlink name. lower contains symlink name for the lower layer.

UpperDir: The read-write layer that represent the changes are part of UpperDir. This is basically a container layer.

[root@cvp95 diff]# ls -la /var/lib/docker/overlay2/3ee2e31d2b5eeafd9cbe6c99bcf1d2f7121c272beebf7a088fede8e564fcd368/diff
total 12
drwxr-xr-x 3 root root 4096 Oct  4 09:57 .
drwx------ 4 root root 4096 Oct  4 09:57 ..
drwxr-xr-x 3 root root 4096 Sep 25 22:34 run

MergedDir: The merged dir is the result both LowerDir and UpperDir that Docker used to run the containers.

WorkDir: This directory is an internal directory to overlayv2 and it is empty.

HOPE YOU LEARNT SOMETHING NEW IN THIS TUTORIAL. IF YOU HAVE ANY DOUBTS FEEL FREE TO COMMENT BELOW. AND STAY TUNED FOR MORE TUTORIALS :)

Comments

Post a Comment

Popular posts from this blog

Lets learn "About kube proxy in iptables mode"

-
Image
In this tutorial, we'll learn about the Kube-proxy in iptables mode. Basically, we'll see how Kube-proxy create iptables that help in load balancing and service discovery. If you are not familiar with Linux IPTables I would recommend to check out my tutorial on IPTables here  first and then come back to read this tutorial. Kube-proxy Kube-proxy when running in iptables mode basically watch kube-apiserver for new endpoints and create corresponding iptables rules and iptables are fully responsible for service discover and proper routing of traffic to corresponding pods. Kube-proxy needs to run on all k8s worker nodes. Below is a small video describing the working of kube-proxy in iptables mode.  src:  https://www.slideshare.net/CJCullen/kubernetes-networking-55835829 Now that we know what Kube-proxy is and how it Kube-proxy works, let deep dive into the types of iptables Kube-proxy adds to the firewall. Kube-proxy added IPTables To learn this let's consider an example. Cons...
Read more

Lets learn "System design for paste bin (or any text sharing website)"

-
Image
  In this tutorial, we'll learn about the system design for PasteBin. If you don't know what Pastebin is here goes its definition. Pastebin is a type of online content hosting service where users can store texts to share with others over the internet.  Pastebin system design requirements  Functional Basic - User should be able to paste the text, generate the link and share the link with other users. There should be the max size of the text that our system will support (say 10Mb) Users should be able to provide the custom URL path for the link created. Like if a user wants to create the links  www.pb.com/letlearn, he can give this URL and pastebin should create a link using this URL to share with others. The link or paste expiry feature should be there for each stored text. There can be default expiry time say 1 year but this should be configurable. Personalization means the user should be able to login to track all the pastes created by him and manage them. Non-Funct...
Read more

Lets learn "What is CDN?"

-
Image
  In this tutorial, we'll learn about What is CDN ( Content delivery network ) and how it works. Introduction Whenever you access any website on your mobile phone or laptop or any other device using the internet connection, you must have seen for some of the websites the pages load quickly, and for others, it takes considerably more time. Why is that?  Basically loading any website requires the data needs to be downloaded from the server where that particular website is hosted. For e.g, if you are accessing a website in Canada which is hosted on a server in India, it will take some time due to the latency but if that website is hosted on servers in Canada itself it will take less time to load due to low latency. So internet experience can be bad for some of the websites when clients and servers are separated by a long distance. One could argue that why not host the website on servers in all regions but that would increase the cost a lot. The better solution is to use CDN. Let'...
Read more